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Members  Present:  Chata  Ashley,  Ari  Bachrach,  Manny  Grueso,  Marcel  Miranda,  Perry  Rhodes,  III, 
Israel  Nieves-Rivera. 

Members  Absent:  Michael  Discepola,  Joani  Marinoff ,  Lisa  Reyes  (HPS). 

Professional  Staff:  Dara  Coan  (Harder  c\  Co),  Kym  Dorman  (Harder  &  Co),  Tracey  Packer  (HPS), 

Mike  Pendo  (HPS),  David  Weinman  (Note  Taker),  and  Allison  Weston  (Harder  & 
Co). 

1.  Welcome  and  Announcements 

Perry  Rhodes,  III  called  the  meeting  to  order  at  1:10  PM.  He  asked  attendees  to  introduce 
themselves,  make  announcements,  and  as  ar\  icebreaker  question  asked, "  What  do  you  plan  to  do 
this  weekend?' 

2.  Public  Comment 

There  was  no  Public  Comment. 

3.  Discuss  Scope  of  Work 

Background 

Dara  Coan  distributed  the  document  entitled,  "A  Brief  History  of  Evaluation  in  San  Francisco, "a 

copy  of  which  is  available  to  absent  members  upon  request. 

•      Tracey  Packer  provided  an  overview  of  the  history  of  Evaluation  in  SF  - 

o    In  1999  an  HPPC  Committee  decided  upon  three  levels  of  evaluation:  providers'  capacity, 

intervention  research,  and  the  measurement  of  impact  (surveillance  data), 
o    In  1999  the  HPPC  also  started  a  Behavioral  Risk  Assessment  (BRA)  project  with 

providers  in  which  behavioral  criteria  is  used  to  compile  data  on  outcomes. 
<=>    And  in  1999  the  CDC  required  funded  jurisdictions  to  implement  client-level  data 
reporting. 

To  comply,  the  HPPC  implemented  ELI  collaboratively  with  the  State  of  California. 
*      The  priority  placed  on  ELI  required  the  HPPC  drop  its  BRA  program. 
Many  other  jurisdictions  flat-out  refused  to  implement  any  evaluation  reporting 
system. 
>=>    2002-03  the  HPPC  contracted  with  CompassPoint  to  provide  OD/TA  for  ELI 

implementation. 
o    In  2003  the  CDC  released  a  new  evaluation  requirement  for  reporting  client-level  data, 
referred  to  as  Program  Evaluation  and  Monitoring  System  (PEAAS). 
PEAAS  is  a  web-based  individual-level  collection  system. 
The  CDC  has  said  that  data  from  ELI  will  be  merged  with  PEMS  system, 
o    In  2003  the  CDC  also  required  evaluation  reporting  based  on  their  Performance 
Indicators  (a  copy  is  included  in  the  " Current  Evaluation  Requirements"  document). 
The  HPS  worked  with  HPPC  members  to  develop  one  and  five-year  baselines  and 
goals. 
■=>    In  2003  the  HPPC  wrote  a  new  multi-year  HIV  prevention  plan. 
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Israel  Nieves-Rivera  noted  that  in  1999  there  was  a  conflict  between  federal  and  state 
authorities  regarding  what  data  was  to  be  collected  and  the  CDC  backed  down  allowing 
states  to  develop  their  own  system. 

California  came  up  with  its  ELI  system,  but  many  states  didn't  implement  a  system  at  all. 
=>    Last  year  the  CDC  mandated  the  use  of  PEMS. 
«=>    He  highlighted  that  local  directly  funded  agencies  are  currently  entering  data  into  both 

systems. 

He  pointed  out  that  PEMS  and  ELI  capture  different  data. 

Evaluation  Requirements 

Dara  distributed  the  document  entitled,  "Current  Evaluation  Requirements,"  a  copy  of  which  is 
available  to  absent  members  upon  request.  She  provided  an  overview  of  the  requirements. 
&    She  noted  that  HPS  contractors  choose  an  intervention  to  evaluate  for  Outcome 
Evaluation. 
•      Tracey  said  that  currently  it  is  unclear  if  the  CDC  requires  Outcome  Monitoring  and 
Outcome  Evaluation,  noting  that  they  are  not  the  same. 

The  HPS  is  meeting  with  representatives  from  CDC  next  week  to  clarify  this  issue. 
Israel  asked  if  the  criteria  for  Outcome  Evaluation  shifted  in  1999  from  behavior  change  to 
other  results  -  such  as  confirmation  of  referral. 
■=>    Tracey  said  that  originally  the  criterion  was  behavioral  and  that  ultimately  changing 

behavior  must  be  the  goal. 
■=>     Dara  noted  that  the  contracting  allows  flexibility  in  outcome  criteria. 
■=>    Israel  suggested  the  committee  discuss  this  as  part  of  its  scope  of  work  this  year. 
Dara  explained  that  the  CDC  also  requires  data  for  performance  indicators  that  may  be 
problematic. 

<*    Surveillance  data,  for  example,  sometimes  came  from  research  that  is  no  longer  funded. 
She  noted  that  Willi  McFarland  pointed  out  the  discontinuation  (due  to  decreased 
funding)  of  much  of  the  research  previously  used  for  HIV  surveillance  in  his  Council 
presentation  01/08/04. 
In  response  to  Allyson's  question,  Tracey  explained  that  non-names  HIV  test  reporting  uses 
people's  initials  and  part  of  their  Social  Security  number  as  a  code/identifier. 
The  code,  she  added,  allegedly  can't  be  traced  back. 

She  also  noted  that  not  all  testing  agencies  are  up  and  running  with  this  system. 
Marcel  Miranda  noted  that  non-names  reporting  started  in  2002,  with  screening 
sites  reporting  monthly. 
<=>     Tracey  and  Dara  suggested  inviting  Sandy  Schwarcz  to  a  Committee  meeting  to  discuss 
non-names  reporting  information. 

Mike  also  suggested  Maree  Kay  to  present  at  a  committee  meeting. 
Israel  suggested  checking  if  California  has  added  a  "  Transgender" category  to  their  non- 
names  specimen  form. 

Dara  highlighted  that  the  Committee's  discussion  underlines  how  unclear  evaluation 
requirements  are. 
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Draft  Scope  of  Work 

Dora  distributed  the  document  entitled, "  HPPC  Evaluation  Committee:  Draft  Scope  of  Work,"  a 

copy  of  which  is  available  to  absent  members  upon  request. 

•  Perry  reviewed  the  Committee  goals  as  outlined  in  the  draft  document.  This  includes 
developing  a  multi-year  strategic  plan  for  evaluation  in  San  Francisco. 

Israel  added  that  if  the  committee  was  to  come  up  with  evaluation  criteria  that  could  be 

included  in  the  2005  RFP  anticipated  for  release  in  2004,  that  work  would  need  to  be  done 

early  in  the  year. 

■=>     He  suggested  the  Committee  schedule  the  year's  work  in  two  parts: 

Jan  to  April  -  work  on  evaluation  requirements  for  the  2005  RFP  Cycle;  and 
May  to  Dec  -  other  specific  goals  for  development  of  prevention  evaluation. 

■=>     Dara  asked  if  the  suggestion  regarding  the  2005  RFP  cycle  would  include  specific 
evaluation  methodology. 

Israel  responded  that  the  RFPs  need  to  ensure  that  applicants  collect  the 
information  needed  for  evaluation  relative  to  both  the  SF  Plan's  objectives  as  well  as 
the  funders'  requirements. 

•  Israel  pointed  out  that  the  committee  does  not  work  in  a  vacuum. 

■=>    In  specific  he  noted  that  the  committee  needs  to  coordinate  with  the  work  of  the 

Prevention  With  Positives  and  New  Approaches  to  Prevention  Committees. 
He  also  cautioned  against  creating  objectives  that  are  too  big  and/or  not  achievable. 

In  way  of  example  he  pointed  to  the  experience  of  the  HPPC's  2002  Research 

Committee  which  established  three  "lofty  goals. 

That  Committee's  work  ultimately  had  to  be  put  into  more  manageable  steps  and  only 

part  was  accomplished  when  the  term  expired. 
Marcel  suggested  some  sort  of  Quid  Pro  Quo  for  agencies  in  which  the  HPPC  would  tell  them 
what  is  wanted  from  them  and  at  the  same  time  what  they  get  for  it. 
Chata  stated  that  agencies  have  been  doing  some  sort  of  evaluation  since  1995. 
<=>     He  asked  if  they  are  reporting  their  data. 

o    Tracey  said  that  they  are,  but  it  is  not  possible  to  aggregate  it  into  one  report. 
<=>     She  added  that  it  might  be  valuable  to  hear  providers'  perspectives  on  data  collection 

and  reporting. 
Chata  asked  if  agencies  are  scored  as  part  of  program  evaluation. 
o    Mike  explained  that  Process  Evaluation  often  involves  a  score  on  their  monitoring 

report. 
<=>     Outcome  Evaluation  or  the  impact  the  program  had  on  the  client,  he  added,  is  more 

difficult. 
Mike  suggested  structuring  the  committee's  goals  as  multi-year  plans  so  as  to  have 
milestones  of  accomplishments  and  implementation. 

<=>    Tracey  added  that  the  first  milestone  could  be  addressing  the  2005  RFP. 
Dara  noted  that  this  is  an  opportunity  to  bring  evaluation  standards  into  line  with  the  goals 
of  the  San  Francisco  Leadership  Initiative  (SFLI). 
■=>    Tracey  added  that  evaluation  should  be  part  of  the  members'  vision  of  prevention  in  SF. 
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Israel  stated  that  there  hasn't  been  a  link  between  funding,  and  programs'  effect  on 
behavior/incidence. 

Dara  added  that  Willi  McFarland's  presentation  asserted  that  incidence  is  leveling  off, 

or  declining,  as  a  result  of  prevention,  but  there  isn't  a  direct,  proven  link. 

Mike  pointed  out  that  although  the  relationship  isn't  directly  proven,  research  has 

established  a  correlation  sufficiently  for  policy  purposes. 

He  added  that  ultimately  the  goal  of  all  prevention  activity  is  to  reduce  incidence. 

Committee  Goals 

Perry  asked  if  the  Committee  would  accept  the  Goals  as  proposed. 

Tracey  suggested  adding  a  specific  time  rather  than, "...  this  year*'  in  the  second  bullet. 
=5     Marcel  suggested  adding  both  specific  times  as  well  as  prioritizing  the  objectives. 
■=>    Mike  suggested  changing  the  first  bullet  by: 

*  Specifying  strategic  plans  of  a  specific  number  of  years,  and 

*  Adding  sub-points  with  specific  goals  by  year(s) 

Tracey  suggested  a  five-year  strategic  plan,  since  2004  is  the  first  year  of  a  five- 
year  CDC  funding  cycle. 
■=>     Dara  noted  that  in  the  near  future  the  committee  will  have  a  better  idea  of  what  CDC's 
guidance  will  require. 

Perry  questioned  if  the  committee's  goals  could  be  counter  to  the  CDC's  guidance, 
o     Tracey  suggested  adding  a  note  that  the  Goals  could  be  amended. 
Perry  asked  the  committee  if  the  Goals,  as  amended,  would  be  accepted  as  the  2004  Scope  of 
Work.   There  was  general  agreement,  and  the  draft  was  accepted. 

Israel  suggested  using  the  first  four  months  of  2004  to  address  Question  1 "  What  is  San 
Francisco  required  to  do  in  terms  of  evaluation,  and  how  can  we  do  it  better?" 
^     He  also  suggested  that  remainder  of  the  year  address  Question  2  "Do  the  evaluation 
activities  we  implement  actually  tell  us  if  we  are  doing  good  work?   What  does  San 
Francisco  want  to  do  in  terms  of  evaluation  to  show  that  the  local  HIV  prevention  model 
works  and  to  identify  areas  for  improvement?  How  do  we  stay  'ahead  of  the  game'?' 
•      Mike  suggested  the  Committee  use  the  first  month  reviewing  and  agreeing  to  its  vision, 
priorities,  process,  and  steps. 

Committee  Process 

The  Committee  discussed  how  to  proceed  with  its  business. 

In  response  to  a  question,  Tracey  stated  that  the  Committee  need  get  approval  from  the 
Steering  Committee  or  the  Council  with  its  Goals  as  authority  has  already  been  given  it  to 
establish  goals  and  a  scope  of  work. 

Elect  Committee  Chair  /  Co-chairs 

Chata  provided  background  on  what  is  involved  in  being  a  Chair  /  Co-Chair,  including  participating 
in  the  Steering  Committee. 

=>     Discussion  followed  regarding  whether  the  Committee  would  have  a  Chair  and  a  Co-Chair 
or  just  a  Chair,  although  consensus  was  not  reached. 
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•  Volunteers  to  serve  as  Chair  and/or  Co-Chair  were  sought. 
<=>    Perry  was  the  only  volunteer. 

It  was  moved  and  seconded  that  Perry  be  the  Evaluation  Committee  Chair.  There  was  no 
further  discussion.  There  were  no  objections  or  abstentions.  The  motion  passed. 

■=>    Israel  offered  to  provide  support  to  Perry,  particularly  if  needed  to  miss  a  meeting. 

Roles  and  Responsibilities  of  Participants 

•  Tracey  provided  explanation  of  Mike's  role  with  the  Committee. 

^     She  noted  that  Mike  is  from  the  Applied  Research  Unit  of  the  HPS  and  that  he  would 

bring  that  expertise  to  the  committee.  He  will  probably  come  to  most,  but  not  all  of  the 

meetings. 
■=>    She  added  that  program  and/or  contract  management  staff  would  be  invited  to  attend 

meetings. 
■=>    She  also  explained  that  she,  Lisa  Reyes,  and  Mike  (when  he  is  in  attendance)  would  share 

a  committee  vote,  with  Lisa  being  the  official  member. 

•  Tracey  also  explained  that  HPS  professional  staff  helps  the  Chair  develop  the  agenda  and 
review  the  minutes. 

Dara  asked  if  the  Committee  wants  to  use  Roberts  Rules  of  Order. 

<=>    Chata  stated  that  committees  haven't  always  used  it. 

<=>    Ari  stated  that  some  means  of  procedure  should  be  articulated  and  used. 

■=>    Tracey  noted  that  the  bylaws  do  not  specify  the  use  of  Roberts  Rules  of  Order,  but  do 

state  that  committees  should  strive  for  consensus. 
<=>    This  item  will  be  on  the  agenda  for  next  committee  meeting  (02/05/04). 

•  Tracey  explained  that  Community  Members  are  Committee  members  who  are  not  on  the 
Council. 

<=>    She  noted  Community  Members  are  sometimes  recruited  to  help  fill  gaps  in  the 
committee's  Parity  And  Representation. 

She  asked  members  to  look  for  people  from  the  community  who  could  serve  on  this 
committee  and  e-mail  any  suggestions  to  either  her  or  Perry. 

<=>    Israel  noted  that  this  is  also  a  good  way  of  recruiting  new  Council  members. 

•  Ari  asked  if  there  are  non-traditional  prevention  providers  that  receive  funding. 

■=>    In  response  to  a  question,  Ari  explained  that " non-traditional provider^'  would  include 
agencies  that  aren't  primarily  HIV  prevention  but  offer  some  prevention  services. 

o    Mike  added  that  it  would  also  be  helpful  to  have  information  on  agencies  that  aren't 
funded,  but  are  looking  to  incorporate  prevention  in  the  program. 

■=>     Dara  said  she  would  email  a  list  of  providers  in  the  BVHP  that  are  doing  prevention. 

o    In  response  to  Marcel's  suggestion,  Tracey  stated  she  would  distribute  a  list  of  agencies 
currently  being  directly  funded. 

<=>    This  item  will  also  be  on  the  agenda  for  next  meeting  (02/05/04). 

5.    Select  regular  meeting  time 

The  Committee  agreed  to  meet  the  first  Thursday  of  each  month,  3:00-5:00  PM. 
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6.  Process  Evaluation  Discussion 

Tracey  explained  that  there  are  several  ways  for  the  Committee  to  provide  input  to  this 
process,  including  Zoomerang,  forms  at  each  meeting,  or  quarterly  meetings  with  a  Process 
Evaluation  consultant. 

There  was  some  discussion  about  providing  feedback  immediately  after  each  meeting  or 

later. 

Israel  noted  that  feedback  shouldn't  be  limited  to  whatever  Process  Evaluation  method  is 

decided  upon;  that  all  feedback  is  important  and  valued. 
Motion  was  made  and  seconded  to  have  evaluation  forms  distributed  at  each  meeting  as  well  as 
quarterly  reviews  with  a  Process  Evaluation  consultant. 

<=>    In  response  to  Israel's  question  about  the  content  of  the  forms  it  was  agreed  to  put 
this  on  the  agenda  for  the  next  meeting  (02/05/04). 
There  was  no  further  discussion.   No  objections  or  abstentions  were  raised.  The  motion  passed 
by  consensus. 

7.  Evaluation  and  Closure 

Evaluation  forms  were  distributed.   The  meeting  adjourned  at  2:20  PM 


THE  NEXT  MEETING  IS  SCHEDULED  FOR  THURSDAY,  FEBRUARY  05,  2004 
FROM  3:00  TO  5:00  PM  -  5™  FLOOR  CONFERENCE  ROOM,  25  VAN  NESS  AVE. 


Minutes  were  prepared  by  David  Weinman,  Reviewed  by  Tracey  Packer,  and  Perry  Rhodes  III. 
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1.    Welcome  and  Review  of  Agenda 


3:00-3:10 


2.  Public  Comment 


3-.10-3.20 


3.  Approval  of  Minutes  from  February  5,  2004 

4.  Report  from  Steering  Committee 


5.   Overview  of  Evaluation 


6.  Review  Priority  Areas 

7.  Begin  Discussion  on  Priority  Areas  1 

8.  Summary,  Evaluation,  and  Closure 


3:20-3:25 
3:25-3:40 
3:40-4:00 
4:00-4:25 
4:25-4:50 
4:50-5:00 


Next  meeting  will  be  April  1 ,  2004  at  3PM 


NOTE:  All  meetings  are  open  to  the  public  and  are  held  In  handicapped  accessible  facilities. 
Meeting  dates  and  times  are  subject  to  change,  please  verify  by  calling  Betty  Chan  Lew  at  554-9492. 

Know  your  rights  under  the  Sunshine  Ordinance:  Government's  duty  is  to  serve  the  public,  reaching  its  decisions  In  full  view  of  thQ 
public.  Commissions,  boards,  councils,  and  other  agencies  of  the  city  and  county  exist  to  conduct  the  people's  business.  This 
ordinance  assures  that  deliberations  are  conducted  before  the  people  and  that  City  operations  are  open  to  the  people's  review.  For 
more  information  on  your  rights  under  the  sunshine  ordinance  or  to  report  a  violation  of  the  ordinance,  contact  the  Sunshine 
Ordinance  Task  Force    Donna  Hall,  City  Hall,  Room  244,  1  Dr.  Carlton  B.  Goodlett  Place,  San  Francisco,  CA  94102,  Phone:  554- 
7724,  Fax:  554-7354,  E-Mail:  Donna_Hall@C'.sf.ca.us. 

In  order  to  assist  with  the  City's  efforts  to  accommodate  persons  with  severe  allergies,  environmental  illnesses,  multiple  chemical 
sensitivity  or  related  disabilities,  attendees  at  public  meetings  are  reminded  that  other  attendees  may  be  sensitive  to  venous 
chemical  based  products.  Please  help  the  City  accommodate  these  Individuals. 


TOTPL   P. 02 
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Members  Present:  Chata  Ashley,  Ari  Bachrach,  Marcel  Miranda,  Perry  Rhodes,  III,  Lisa  Reyes 
(HPS),  and  Frank  Strona. 

Members  Absent:  Michael  Discepola,  Manny  Grueso,  Israel  Nieves-Rivera. 

Professional  Staff:   Dara  Coan  (Harder  &  Co),  Kym  Dorman  (Harder  &  Co),  Tracey  Packer  (HPS), 
Mike  Pendo  (HPS),  Marise  Rodriguez  (HPS),  David  Weinman  (Note  Taker),  and 
Allison  Weston  (Harder  &  Co). 

1.     Welcome  and  Announcements 

Perry  Rhodes,  III  called  the  meeting  to  order  at  3:06  PM.   He  asked  attendees  to  make  any 
relevant  announcements. 

Tracey  Packer  announced  that  Frank  Strona  would  like  to  join  this  Committee. 

Committee's  Procedure 

Perry  opened  this  topic  by  noting  the  discussion  from  the  Committee's  previous  meeting 

(01/22/04).   He  pointed  out  that  members  discussed  the  values  and  limitations  of  both: 

Robert's  Rules  of  Order  (Robert's),  and  decisions  by  consensus.   He  reminded  members  of  the 

consensus  reached  to  think  about  this  topic  between  meetings  and  make  a  decision  at  this 

meeting. 

•  Frank  stated  that  the  New  Approaches  to  Prevention  Committee  had  a  similar  discussion 
and  agreed  to  use  the  less  formal  consensus  method  unless  there  was  controversy,  in  which 
case  votes  will  be  conducted  in  accordance  with  Robert's. 

•  Tracey  discussed  the  idea  of  consensus  as  a  decision  all  members  could  accept. 

<z>     She  added  that  in  this  case  efforts  should  be  made  to  accommodate  all  members'  views, 
although  there  could  be  circumstances  in  which  the  majority's  view  may  need  to  prevail. 

•  Marcel  Miranda  stated  that  reaching  consensus  could  be  trusted  to  the  Chair. 


Proposal  was  made  for  the  Committee's  business  to  be  conducted  informally,  and  to  vote  using 
Robert's  only  when  consensus  couldn't  be  reached.  The  proposal  was  accepted  by  consensus. 


Approval  of  Minutes  from  the  01/27/04  Meeting 

Perry  asked  if  the  Minutes  as  submitted  were  approved  by  the  Committee,  or  if  there  are 
corrections  and/or  clarifications. 

Allison  Weston  noted  her  name  misspelled. 


There  were  no  other  changes.   The  minutes  were  approved  without  abstention. 


Public  Comment 

None 

Discussion  of  Data  Collection  Systems  (added  to  agenda  to  meeting) 

Perry  noted  the  email  sent  by  Dara  Coan  regarding  the  ELI  and  PEMS  systems.   He  explained 
that  Marise  Rodriguez  was  invited  to  the  meeting  to  clarify  the  differences  between  the 
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systems.   Manse  distributed  the  document  entitled,  "Evaluation  and  Data  Collection  Issues  at 
hand,"  a  copy  of  which  is  available  to  absent  members  upon  request. 

Manse  stated  that  to  completely  merge  the  data  from  any  two  systems  the  variables  must 

be  identical,  but  that  the  variables  in  the  ELI  and  PEMS  systems  are  not  alike. 

In  addition  to  the  topics  listed,  Marise  noted  other  issues  including  the  following. 

■=>    Locally  two  years  of  work  has  gone  into  ELI,  including  implementation  and  training. 

o    In  concept,  there  are  advantages  to  PEMS  over  ELI  such  as. 

It  includes  scanning  of  data  from  forms  directly  into  system, 

Based  on  design  specifications,  it  seems  to  be  easier  to  use  than  ELI, 

Use  of  newer,  more  advanced,  programming  tools,  and 

Flexibility  in  the  way  variables  are  input  and  the  customizable  formats  used. 

■=>    In  the  event  PEMS  is  ultimately  used,  there  would  be  three  ways  to  input  data: 

1.  Direct  input  on  CDC  computer 

*  Users  maintain  direct  connectivity  and/or  use  the  Internet; 

2.  Local  system 

*  Users  connect  with  locally  maintained  system  which,  in  turn,  periodically  shares 
data  with  the  CDC  system;  and 

Chata  noted  that  it  isn't  known  when  PEMS  will  actually  go  on  line. 

He  questioned  if  ELI  could  be  modified  to  interface  well  with  PEMS. 

Chata  then  asked  if  the  CDC  could  force  SF  to  use  PEMS. 

^     Marise  explained  the  CDC  could  force  SF  to  use  it,  particularly  if  the  State  isn't  moving 

toward  making  ELI  compatible. 
■=>    There  was  general  agreement  that  the  CDC  could,  indeed,  compel  SF  to  use  PEMS. 
Marcel  stated  that  some  things  are  exciting  about  PEMS,  he  suggested  getting  community 
input. 
<=>     He  pointed  out  that  this  would  also  demonstrate  that  SF  is  attempting  to  understand 

providers'  perspectives  on  ELI. 
■=>     Once  the  community's  view  is  obtained,  he  added,  SF  HPPC  should  actively  advocate  one 

way  or  the  other. 
Frank  stated  that  small  agencies  find  it  difficult  to  comply  with  ELI. 
He  noted  that  the  people  designing  PEMS  (De/o/tte)  are  programmers,  without  specific 
understanding  of  prevention  and  prevention  programs. 
o    He  added  that  in  his  experience  such  systems  often  roll-out  with  significant  flaws 

requiring  updating. 
Chata  stated  that  HPPC  needs  to  be  careful  going  to  the  community;  HPPC  needs  to  know  all 
of  what  is  involved  before  stirring  up  questions  and  concerns. 
Dara  asked  about  the  long-term  administrative  costs  of  dealing  with  two  systems. 

She  pointed  out  that  costs  of  not  transitioning  include:  modifying  ELI  to  be 

compatible  with  PEMS,  and  continual  updating  it  to  keep  the  systems  in  sync. 

She  noted  that  SF  and  STATE  might  have  to  decide  before  seeing  ar\  operational 

PEMS  system. 
■=>    In  response,  Marise  stated  that  the  variables  are  established  although  the  business 
rules  haven't  been  settled  as  yet. 
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Dara  also  stated  that  ELI  is  not  user  friendly  and  will  always  require  a  lot  of  user  training. 

Marise  noted  the" Primary  Partner*'  is  not  included  in  PEAAS. 

o    Marise  said  that  she  has  raised  the  importance  of  field  with  the  CDC. 

■=>     Mike  Pendo  stated  that  although  perhaps  relevant  for  counseling,  for  reporting  purposes 

"  Primary  Partner"  isn't  important,  as  long  as  HIV  status  is  known. 
■=>    Ari  asked  if  PEMS  includes  data  on  co-factors. 

He  added  if  it  does,  the  serostatus  of  partner  is  relevant  to  reporting. 
o    Mike  stated  that  the  purpose  of  the  system  is  to  track  risks  of  transmission. 
Tracey  stated  that  the  choice  of  which  system  to  use  is  probably  not  the  HPPC's. 

She  noted  that  both  STATE  and  SFDPH  seem  to  be  leaning  toward  staying  with  ELI. 
o    Marise  stated  that  we  should  have  a  plan  in  case  we  have  to  go  with  PEMS. 
Marise  stated  that  while  ELI  has  been  problematic,  the  experience  is  that  changes  have 
been  made  when  suggested. 

<=>     She  added  larger,  more  remote,  bureaucracies  will  probably  not  be  as  responsive. 
Frank  stated  that  this  is  a  huge  cost  for  providers  and  that  once  a  cost  factor  is 
determined  accommodation  should  be  included  in  RFPs. 
■=>     He  added  that  there  are  many  problems  with  ELI,  including  its  inability  to  provide  client 

satisfaction  data. 
Ari  asked  if  either  system  actually  provides  evaluation  data. 
o    Marise  stated  that  it  takes  a  lot  of  requests  to  get  custom  reports  from  ELI. 
o     Ari  asked  if  SF  would  have  more  clout  in  design  and  implementation  if  it  started  working 

with  PEMS  now. 
o    Marise  responded  that  she  and  Charles  Klein  have  been  working  with  the  PEMS 

designers. 
^>     She  noted,  however,  that  discussion  regarding  the  variables  closed  01/30/04. 
Chata  said  that  burdening  agencies  with  data  collection  distracts  them  from  providing 
services. 

o     He  asked  if  there  are  plans  to  recompense  for  data  collection. 
Dara  noted  that  SF  could  add  variables  to  PEMS,  which  is  nearly  impossible  with  ELI. 
She  highlighted  that  scanning  forms  into  a  system  is  a  huge  benefit/cost  saver. 
Chata  asked  about  costs. 
<=>    Marise  responded  that  SF  currently  spends  about  $75K  annually  for  ELI  maintenance 

and  training,  and  that  PEMS  could  add  to  this  figure  more. 
o    It  was  noted  that  there  would  also  be  recurring  costs  of  keeping  ELI  in  sync  with  PEMS. 
Marise  suggested  sending  a  letter  to  the  STATE  telling  them  that  forms  should  be 
scannable. 
o    She  also  noted  that  there  is  a  series  of  e-mails  requesting  changes  she  has 

recommended  for  the  ELI  system  that  have  not  yet  been  acted  upon  due  to  State's 

competing  priorities  (ELI  &  CTR.) 
Frank  suggested  investigating  what  the  providers  did  and  didn't  get  from  ELI  -  is  it  working 
for  them,  or  isn't  it  -  perhaps  run  a  'SCAN  type  process, 
o    The  Committee,  he  added,  needs  to  determine  the  providers'  viewpoint  so  that  it  can 

make  a  recommendation  to  the  Council. 
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Chata  asked  if  a  decision  could  really  be  made  locally. 

Tracey  said  that  if  the  DPH  chose  PEMS  it  could  be  implemented,  but  would  be  complex. 
She  reiterated  the  history  of  building  the  client-level  system  in  partnership  with 
STATE  so  as  to  be  in  compliance  with  the  CDC's  1999  requirements. 

The  CDC,  she  emphasized,  is  now  adding  new  type  of  system  to  meet  its 
requirements. 
Marcel  noted  there  has  been  discussion  regarding  the  value  of  scam'\r\g  or  capturing  data  as 
it  is  collected,  including  while  doing  outreach. 
The  Committee  expressed  their  thanks  to  Marise  for  her  presentation.   It  was  noted  that  her 
document  is  also  colorful  and  attractive. 

Develop  Evaluation  Strategic  Plan 

Kym  Dorman  distributed  the  document  entitled, " HPPC 'Evaluation  Committee:  Draft  Scope  of 
Work,"  a  copy  of  which  is  available  to  absent  members  upon  request.  She  reviewed  the  changes 
to  the  draft  document  from  the  previous  version.  Perry  asked  if  the  draft  reflects  members' 
recollection,  and  also  asked  for  comments. 

Perry  began  the  discussion  by  suggesting  the  Scope  of  Work  be  " Slimmed  down." 

PHASE  ONE 

Vision,  Priorities,  Process,  and  Steps 

Tracey  suggested  the  Committee  discuss  what  materials  it  would  want  to  review  at  the 

March  meeting  (03/04/04). 

o     She  noted  that  there  was  an  evaluation  strategic  plan  in  the  1997  Plan. 
Strategic  Questions  for  the  RFP 

An  expressed  difficulty  with  establishing  strategic  questions  for  the  next  RFP,  as  the 

deadline  is  near  and  such  informs  the  whole  funding  cycle. 

Marcel  agreed  with  Ari,  adding  that  the  Committee  may  need  more  meetings,  or  a  different 

work  plan  -  such  as  the  use  of  subcommittees  to  draft  proposals. 

Chata  suggested  the  Committee  review  what  has  been  done,  what  is  in  the  Plan,  and 

determine  how  that  has  worked. 

Dara  noted  that  the  RFP  usually  includes  questions  about  how  providers  intend  to  meet  the 

strategic  objectives. 

■=>     She  added  that  strategic  questions,  data  collection,  and  performance  indicators  are 
closely  connected  thus  are  prioritized  together  in  the  Phase  One. 

Tracey  asked  to  delete  the  reference  that  the  RFP  will  be  released  in  May. 


The  Committee  agreed  to  delete  reference  to  the  May  date  for  distribution  of  the  2005  RFP. 


Provider  Data  Collection 
Streamline  Requirements 
■=>    Ari  expressed  the  need  to  know  what  is  currently  required  of  providers,  adding  that  he 

doesn't  want  to  make  providers  jump  through  more  hoops. 
■=>     Dara  noted  that  providers  often  see  evaluation  as  obstacles,  or  "hoops,"  but  it  should  be 
a  helpful  tool  for  them. 

She  added  that  making  it  useful  for  providers  is  part  of  the  committee's  purpose. 
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■=>     Ari  stated  that  the  usefulness  of  evaluation  depends  upon  who  collects  and  interprets 
the  data. 

Front  line  staff,  he  added,  may  see  "hoops"  while  program  managers  see  the  value. 
=>    Mike  stated  that  the  way  to  make  evaluation  valuable  to  providers  at  all  levels  is  to  be 

sure  to  get  data  back  to  them. 
■=>     Frank  noted  that  providers  have  been  frustrated  by  not  getting  the  information  they 

want/need  from  ELI. 
■=>    Mike  suggested  compiling  a  list  of  requirements  in  providers'  contracts  compared  with 

what  is  actually  being  done. 
^    Tracey  provided  a  brief  overview  of  current  evaluation  requirements:    1)  ELI;  2) 

outcome  objective  for  one  intervention;  3)  client  satisfaction  assessment. 
■=>    Ari  asked  about  the  criteria  to  determine  outcomes. 

Frank  explained  that  Walden  House  uses  25  questions  from  ELI  and  adds  25 
program-specific  to  determine  both  outcomes  and  client  satisfaction. 
He  added  that  an  outcome  could  be  a  client's  "intent/on  to  use  a  condom." 
Provider  Input 

■=>    Marcel  stated  that  it  would  take  time  to  set  up  a  meeting  with  providers  and  wondered 
if  it  could  be  done  within  the  time  restraints  of  Phase  One  as  drafted. 

PHASE  TWO 

Perry  suggested  looking  at  the  steps  in  Phase  Two  and  choosing  three. 
Quality  Assurance 

Ari  suggested  prioritizing  Quality  Assurance. 
Program  Effectiveness 

Frank  suggested  prioritizing  " How  Providers  Can  Use  the  Evaluation  Processes"  rather  than 

determining  Program  Effectiveness. 

■=>     He  cited  several  examples  where  this  would  be  valuable  to  providers. 

It  was  suggested  to  include  Quality  Assurance  and  Updates  on  the  Epidemic  as  part  of 

Program  Effectiveness. 


The  committee  agreed  to  prioritize  the  use  of  evaluation  processes/program  effectiveness  with 
the  understanding  that  the  wording  will  be  worked  out  later. 

Effectiveness  of  HPPC's  Planning  Efforts 

It  was  suggested  to  either  eliminate  this  issue  or  deal  with  it  by  way  of  a  short  written 

explanation. 
Cost-effectiveness  Model 

Ari  stated  the  cost-effectiveness  model  could  be  problematic  for  the  SF  HPPC  in  the  face 

of  the  current  national  trends  from  CDC  and  the  administration. 

Mike  suggested  the  committee  come  up  with  general  language  for  an  overall  evaluation  of 

cost-effectiveness,  rather  than  attempting  to  create  a  mathematical  formula. 

Chata  stated  that  we  should  look  at  whether  one  agency  is  spending  more  for  case 

management  than  others,  and  ask  why. 

Members  agreed  that  this  is  probably  too  much/too  big  to  work  on  during  this  year. 
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Mike  stated  that  he  could  write  something  up  on  Cost  Effectiveness  and  send  it  off  to 
members. 

The  Committee  agreed  to  deal  with  the  cost-effectiveness  model  by  way  of  language  describing 
overall  evaluation  (provided  by  Mike).  It  was  also  agreed  to  discuss  members'  ideas  on  this  at  a 
future  meeting. 

Referral  Tracking 

Frank  suggested  prioritizing  referrals  and  collaboration  as  it  enhances  effectiveness. 
«=>    It  would  be  great,  he  added,  if  the  committee  could  come  up  with  a  methodology  for 
agencies  to  track  referrals. 

The  Committee  agreed  to  prioritize  Developing  Referral  Tracking. 

Needs  Assessment 

Ari  suggested  removing  Needs  Assessment  from  the  Committee's  scope  of  work  for  2004. 
•      Tracey  suggested  Needs  Assessment  is  a  requirement  of  the  priority  setting  model  but 

could  be  a  review  of  the  gaps  in  research  data. 

Mike  suggested  members  review  the  data  on  linkages  from  the  1997  Plan. 

■=>    Copies  of  that  part  of  the  Plan  will  be  emailed  to  members. 


The  Committee  agreed  to  discuss  this  topic  further  after  reviewing  the  linkages  section  of  the 
1997  Plan. 


Present  Updates  on  the  Epidemic 

•  Perry  suggested  including  Updates  with  Quality  Assurance. 

Ari  asked  why  this  is  the  Evaluation  Committee  presenting  Updates  on  the  Epidemic. 
Chata  stated  that  Willi  McFarland  provides  excellent  updates  to  the  Council  twice  a  year. 

•  Dara  pointed  out  that  the  Council  agreed  to  have  Updates  on  the  Epidemic  twice  yearly  with 
the  understanding  that,  when  necessary,  it  would  take  action  based  of  those  updates. 
Tracey  stated  that  this  is  a  way  to  monitor  the  changes  in  epidemic. 

Summary 

•  Ari  suggested  three  priorities: 

1.  Program  Effectiveness  (including  Quality  Assurance), 

2.  Referral  Tracking,  and 

3.  Addressing  Population  Needs  (including  Needs  Assessment  and  Updates  on  the 
Epidemic). 

■=>    He  noted  that  an  overall,  written  summary  would  address  Effectiveness  of  HPPC 
Planning  and  Cost-effectiveness. 


The  Committee  reached  consensus  on  these  three  priorities. 

Next  Steps 

It  was  suggested  that  during  the  Committee's  next  meeting  (03/04/04)  members  discuss, 
and  hopefully  reach  consensus,  on  of  the  bullets  from  the  draft  of  Phase  One. 
It  was  also  suggested  the  Committee  continue  discussion  on  the  Data  Collection  issue. 
Tracey  suggested  the  Committee  develop  an  overall  strategic  plan. 

■=>     Frank  agreed  with  the  suggestion  adding  that  members  need  some  training  and  mutually 
agreed  definitions  regarding  the  Strategic  Planning  process. 
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He  stated  that  members  need  to  be  sure  they  everyone  is  using  the  same  language. 
<=>     Kym  stated  that  she  has  background  in  Strategic  Planning  and  volunteered  to  provide  an 
overview. 


The  Committee  agreed  to  review  the  Strategic  Planning  process  at  a  future  meeting. 


4.    Process  Evaluation  Discussion 

•      Tracey  reminded  members  that  it  had  agreed  to  use  a  paper-based  evaluation  process  but 
had  not  yet  determined  the  questions. 

o    She  circulated  examples  of  forms  previously  used  by  the  HPPC. 
Frank  suggested  including  the  question:  "  Did you  prepare  adequately  for  the  meeting?" 

He  noted  that  such  was  used  successfully  by  the  Plan  Research  Committee  in  2003. 
■=>     An  asked  if  that  means  you  read  the  material,  or  did  other  background  research. 

Frank  responded  that  the  question  refers  to  whatever  is  appropriate. 


The  Committee  reached  general  agreement  to  include  that  question. 


Frank  noted  that  a  question  used  last  year  asking,  "How  well  the  meeting  was  facilitated" 
may  have  contributed  to  the  resignation  of  a  Committee  Co-chair  the  role  of  facilitator. 

Committee  members,  he  added,  need  to  demonstrate  sensitivity  to  each  other. 
o     Kym  suggested  the  question  be  open-ended. 
■=>     Ari  asked  if  criticism  offered  of  a  facilitator  is  acted  on  by  the  HPS'  professional  staff. 

He  was  told  that  there  was  such  support. 
Someone  suggested  questions  like, "  What  do  you  need  for  the  next  meeting?' ,  "What  went 
well  for  you?",  "What  could  be  improved  on?",  "What  do  we  need  for  next  time?" 
Tracey  will  send  the  questions  to  the  Process  Evaluation  team  for  who  will  put  together  a 
form. 


The  Committee  reached  general  agreement  to  send  the  questions  to  the  Process  Evaluation 
team  for  development  of  a  form. 


5.    Evaluation  and  Closure 

The  meeting  adjourned  at  4:40  PAA 


The  next  meeting  is  scheduled  for  Thursday,  March  4,  2004 

FROM  3:00  TO  5:00  PM  -  25  VAN  NESS  AVENUE,  SUITE  330B,  SAN  FRANCISCO 


Minutes  were  prepared  by  David  Weinman,  Reviewed  by  Tracey  Packer,  and  Perry  Rhodes. 
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AGENDA 


1.    Welcome  and  Review  of  Agenda 


2.   Public  Comment 


3:00-3:10 
3:10-3:20 


3.  Approval  of  Minutes  from  February  5,  2004 

4.  Report  from  Steering  Committee 


5.   Overview  of  Evaluation 


3:20-3:25 
3:25-3:40 
3:40-4:15 


6.  Brainstorm  Ideas  for  Strategic  Plan 

7.  Summary,  Evaluation,  and  Closure 


4:15-4:50 
4:50-5:00 


Next  meeting  will  be  April  1,  2004  at  3PM 


NOTE:  All  meetings  are  open  to  the  public  and  are  held  in  handicapped  accessible  facilities 
Meeting  dates  and  times  are  subject  to  change,  please  verify  by  calling  Betty  Chan  Lew  at  554-9492 

Know  your  rights  under  the  Sunshine  Ordinance    Government's  duty  is  to  serve  the  public,  reaching  its  decisions  in  full  view  of  the 
public    Commissions,  boards,  councils,  and  other  agencies  of  the  city  and  county  exist  to  conduct  the  people's  business.    This 
ordinance  assures  that  deliberations  are  conducted  before  the  people  and  that  City  operations  are  open  to  the  people  s  review.  For 
more  information  on  your  rights  under  the  sunshine  ordinance  or  to  report  a  violation  of  the  ordinance,  contact  the  Sunshine 
Ordinance  Task  Force.  Donna  Hall,  City  Hall,  Room  244,  1  Dr.  Carlton  B.  Goodlett  Place,  San  Francisco,  CA  94102,  Phone:  554- 
7724,  Fax:  554-7854,  E-Mail:  Donna_Hall@ci.sf.ca. us. 

In  order  to  assist  with  the  City's  efforts  to  accommodate  persons  with  severe  allergies,  environmental  illnesses,  multiple  chemical 
sensitivity  or  related  disabilities,  attendees  at  public  meetings  are  reminded  that  other  attendees  may  be  sensitive  to  various 
chemical  based  products    Please  help  the  City  accommodate  these  individuals. 
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Members  Present:  An  Bachrach,  Manny  Grueso,  Marcel  Miranda,  Lisa  Reyes  (HPS),  Perry  Rhodes, 
III,  and  Frank  Strona. 

Members  Absent:  Chata  Ashley,  Michael  Discepola,  and  Israel  Nieves-Rivera. 

Professional  Staff:  Dara  Coan  (Harder  A  Co),  Kym  Dorman  (Harder  &  Co),  Tracey  Packer  (HPS), 
Mike  Pendo  (AIDS  Office),  and  David  Weinman  (Note  Taker). 

1.  Welcome  and  Announcements 

Perry  Rhodes,  III  called  the  meeting  to  order  at  3:15  PM.   He  invited  attendees  to  introduce 
themselves  and  make  relevant  announcements. 

Frank  Strona  announced  that  his  office  has  moved  to  1885  Mission  Street. 

•  He  also  announced  that  Walden  House  has  renamed  their  prevention  efforts  to  Health 
Prevention  Services.   The  program  includes  HIV  prevention  as  well  as  a  wide  range  of  health 
services  for  TB,  Hepatitis  and  the  like. 

Frank  pointed  out  that  the  Council  will  be  at  Walden  House  03/11/04. 

•  Tracey  also  announced  that  the  March  Council  meeting  would  focus  on  MTF  and  the  April 
meeting  on  the  FTM  Transgender  populations. 

•  David  Weinman  announced  that  he  will  not  be  available  to  take  notes  at  the  next  Committee 
meeting  04/01/04. 

Perry  pointed  out  a  review  of  the  Committee's  evaluation  form  has  been  added  to  the  agenda. 

2.  Public  Comment 

There  was  no  public  comment. 

3.  Approval  of  Minutes  from  February  5,  2004 

Perry  asked  why  most  comments  are  attributed  to  specific  people  while  others  do  not 

include  attribution. 

<=>     David  explained  that  he  tries  to  note  who  makes  each  comment,  but  that  sometimes  the 
discussion  is  so  fast  he  doesn't  catch  and/or  note  who  made  each  comment. 

Marcel  Miranda  stated  that  he  reviewed  the  notes  and  found  them  to  be  very  complete  and 

comprehensive. 

■=>    Other  members  joined  him  in  thanking  the  Note  Taker. 
Motion  was  made  and  seconded  to  approve  the  minutes.  There  was  no  further  discussion,  or 
objections.  The  minutes  were  approved  by  consensus. 

4.  Report  from  Steering  Committee 

Frank  provided  ar\  overview  of  the  latest  Steering  Committee  meeting. 

Kathleen  Roe  of  the  Process  Evaluation  Team  provided  a  very  positive  report  of  the 
February  meeting,  including  comments  about  the  Council's  methods  and  procedures  being 
solidly  in  place  while  at  the  same  time  maintaining,  "Heart" 

There  was  discussion  regarding  guest  speakers,  including  the  presentations  at  the  last 
Council  meeting  02/12/04. 
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It  was  suggested  that  members  receive  some  training,  or  direction,  on  dealing  with 

statements  they  disagree  with. 

One  way  of  handling  such  disagreement,  without  confrontation,  could  be  to  pass  a 
note  to  the  Co-Chairs  and  let  them  use  their  facilitation  skills. 
■^     Another  suggestion  was  to  provide  handouts  that  familiarize  guests  with  the  Council's 

scope.   Tracey  mentioned  that  the  council  already  has  a  protocol  in  place,  which  was 

developed  by  the  2002  Research  Committee. 
Meeting  agendas  for  March  and  April  were  reviewed,  including  the  need  to  have  ample  time 
to  discuss  the  MTF  and  FTM  Transgender  populations  by  having  separate  agenda  items 
during  two  meetings. 
Questions  and  comments  followed. 

Marcel  stated  that  he  wasn't  clear  why  Supervisor  Duf  ty  was  part  of  the  02/12/04  agenda, 
o     He  added  that  perhaps  a  Speakers'  Panel  could  also  set  expectations  regarding  guests, 
o     Frank  stated  that  speakers  need  to  understand  what  the  Council  is  interested  in  and 

what  it  is  empowered  to  address. 
•      Ari  Bachrach  observed  that  ten  years  ago  Council  members  would  have  been  aggressive 
toward  Supervisor  Dufty,  but  that  they  are  more  moderate  now. 
o    He  added  that  perhaps  members  recognize  that  it  would  be  more  valuable  to  have  him  on 

the  side  of  supporting  prevention  services. 
Tracey  pointed  out  that  members  could  make  comments  and  recommendations  directly  to 
guest  speakers. 

Mike  Pendo  stated  that  past  Research  Committees  had  developed  guidelines  for  guest 
speakers. 

■=>    Tracey  added  that  such  guidelines  are  in  place. 
■=>    Marcel  added  that  in  the  2003  Community  Liaison  Committee  established  guidelines  for 

people  wanting  to  address  community  meetings. 

Overview  of  Evaluation 

Kym  distributed  the  document  entitled,  "A  Bit  about  Evaluation  ..."  a  copy  of  which  is  available  to 
absent  members  upon  request.   She  explained  that  it  includes  her  view  of  what  Evaluation 
entails.   She  summarized  Evaluation  as, '"  What  do  we  know  and  what  don't  we  know  about  the 
what  we  have  been  doing?  What  we  haven't  been  doing,  and  what  would  we  like  to  be  doing'." 
She  noted  that  some  members  have  experience  in  evaluation  methodologies,  but  that  the 
Committee  has  expressed  the  desire  to  break  the  process  down  for  purposes  of  explanation  and 
clarity.  In  addition  to  the  information  contained  in  the  document  distributed,  she  provided  an 
overview  as  outlined  below. 

Why  Evaluate? 

Funding  organizations  often  require  some  form  of  evaluation. 

The  San  Francisco  Leadership  Initiative  can  be  built  on,  in  part,  from  information  generated 

by  the  programs  it  supports. 

Mike  added  that  evaluation  is  also  about  programs  asking  how  they  can  do  better. 

What  is  evaluation? 
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Descriptive  explanations  of  Evaluation  were  quoted  from  both  the  2004  and  1997  Plans. 
Evaluation  can  be  viewed  as  including  three  levels:  Provider,  City-wide,  and  Intervention 
Research  level. 

Kym  noted  that  at  its  last  meeting,  02/05/04,  the  Committee  prioritized  three  topics: 

1.  Quality  Assurance, 

2.  Referral  Tracking,  and 

3.  Cost  Effectiveness  /  Program  Assessment. 

She  reviewed  the  current  evaluation  requirements  as  outlined  on  the  distributed  document 

distributed.   A  discussion  of  evaluation  followed. 

|  Please  note  that  the  following  outline  corresponds  to  the  distributed  document's 
i  "Type  of  Evaluation"  for  topic  order.  The  members'  comments  are  thus  not 
necessarily  reported  in  the  same  sequence  as  discussed. 

Needs  Assessmeni 

An  stated  that  Needs  Assessments  tend  to  be  too  small  a  sampling  for  a  broad,  or  City- 
wide  view. 

■=>    He  added  that  the  sampling  from  some  communities  is  so  small  as  to  call  into  question 
the  validity  of  the  reported  data. 

Process  Evaluation 

Including  Quality  Assurance  and  Referral  Tracking 

Ari  asked  what  Referral  Tracking  includes. 

■=>    Mike  explained  that  outcomes  have  relied  on  changes  in  behavior,  but  that  it  might  be 

valuable  to  determine  if  clients  actually  made  appointments  they  were  referred  to. 
Ari  remarked  that  it  is  a  subset  of  process  evaluation. 
■=>     Dara  stated  that  it  could  be  a  process  or  an  intervention,  such  as  in  the  case  of 

referrals  from  outreach. 
Marcel  noted  that  Glide  always  looks  at  how  many  clients  went  to  the  referral. 
•      Mike  added  that  tracking  can  also  look  at  whether  the  referral  was  helpful  to  the  client. 
Ari  questions  whether  Referral  Tracking  should  be  looked  at  as  part  of  Process  Evaluation, 
or  Outcome  Monitoring/Evaluation. 
<=>     Dara  stated  that  there  isn't  always  a  clear  distinction  between  Process  and  Outcome 

evaluations. 

In  way  of  example  she  cited  the  AIDS  Office's  contract  which  has  a  section  on 
"Process  objectives"  with  questions  such  as, x How  many  referrals  did  the  agency 
make  to  Drug  treatment}' 
o    Tracey  stated  that  Referral  Tracing  becomes  Outcome  Monitoring  when  the  success  of 

the  referral  is  evaluated. 
Frank  stated  that  there  are  a  number  of  aspects  to  referrals  that  should  be  tracked, 
including  whether  staff  made  the  right  referral,  and  if  the  client  was  helped. 
Mike  noted  that  several  years  ago  work  was  done  on  what  constitutes  a  successful  referral. 
<=>     He  added  that  the  Council  spent  months  on  the  topic. 
<=>     He  suggested  copies  of  the  findings  be  distributed  to  Committee  members. 

Outcome  Monitoring  and/or  Evaluation 

3 


HIV  PREVENTION  PLANNING  COUNCIL  (HPPC) 

Evaluation  Committee 

Action  Minutes  From  Meetings: 

March  4,  2004 


Including  Cost  Effectiveness  /  Program  Assessment 

•  Dora  Coan  explained  that  "Outcome  Monitoring"  a  new  term  which  is  different  from  outcome 
evaluation  that  involves  a  specific  study  on  the  intervention. 

Frank  asked  if  there  has  ever  been  a  system  to  determine  if  Outcome  Monitoring  of  the 

contract  was  helpful. 

An  stated  that  there  might  be  greater  cost  effectiveness  addressing  groups  other  than 

gay  white  men,  as  that  population's  incidence  has  remained  somewhat  level. 

■=>     He  suggested  that  there  might  be  greater  efficiency  in  addressing  populations  with 
emerging  rates  of  incidence. 

Kym  asked  how  to  track  the  impact  of  spending  on  a  City-wide  level. 

Surveillance  -  was  not  specifically  discussed. 

Qualitative  and  Quantitative  Evaluation  Methodologies 

Kym  pointed  out  that  evaluation  data  is  either  Qualitative  or  Quantitative.  These  'typed  of 
Evaluations  while  not  part  of  the  nA  Bit  About  Evaluation..."  document  was  discussed  at  some 
length  as  summarized  below. 

Quantitative 

■=>    Quantitative  data,  she  explained,  deals  primarily  with  the  numbers  of  people  affected. 

Information  is  collected  by  surveys,  or  from  systems  such  as  ELI. 

She  added  that  the  strengths  are  that  it  provides  statistics. 
<z>    Mike  noted  that  Quantitative  provides  information  on  what  is  going  on,  but  not  why. 
■=>     Frank  added  that  Quantitative  is  formula  based. 
o     Dara  stated  that  a  survey  using  a  grading  scale  is  Quantitative. 
<=>     Frank  stated  that  most  of  evaluation  of  prevention  in  SF  has  been  Quantitative. 
■=>    Ari  observed  that  the  Council  allocates  funding  to  BRPs  based  on  Quantitative  data, 
o    Perry  stated  that  one  of  the  reasons  policymakers  prefer  Quantitative  data  is  so  they 

don't  have  to  make  judgment  decisions. 
<*    Frank  cited  as  an  example  of  the  relative  value  given  to  the  types  of  data  the  recent 
Council  discussions  of  a  proposed  Needs  Assessment  of  Southwest  Asian  &  Northern 
Africa  (SWANA)  population. 

He  suggested  that  if  the  bulk  of  data  presented  had  been  Quantitative  rather  than 

Qualitative  the  Council  probably  would  have  proceeded. 
o    Marcel  stated  that  over  the  past  couple  of  years  the  Council's  allocation  of  resources 
has  been  on  the  basis  of  Quantitative  data  reflecting  the  populations  most  impacted. 

He  pointed  out  that  the  recent  allocations  corrected  previous  allotments  in  which 

funding  did  not  coincide  with  the  number  of  people  affected  in  various  populations. 

He  questioned  if  the  pendulum  is  beginning  to  swing  back  the  other  way. 

Qualitative 

o     Frank  stated  that  Qualitative  evaluation  data  provide  a  wider  picture  of  what  it  going  on 
and  is  often  based  on  anecdote,  or  experience. 

He  cited  HPPC's  Needs  Assessments  and  SCAhis  as  being  Qualitative. 
■=>     Dara  stated  that  a  survey  which  relies  on  comments  to  gather  data  is  Qualitative. 
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<=>     Dara  suggested  that  HIV  prevention  may  have  been  too  dependent  on  Quantitative. 

She  suggested  the  committee,  and  the  SF  HPPC,  take  the  lead  in  looking 

Qualitatively  at  the  causes  of  infection. 

She  also  noted  that  Qualitative  data  has  not  enjoyed  the  same  respect  from 

policymakers  and  funding  organizations  as  Quantitative  data. 
<=>    Mike  stated  that  people  don't  respond  to  numbers  as  much  as  to  individual  stories. 

He  added  that  SF  has  plenty  of  Quantitative  data  (numbers),  but  little  persuasive 

Qualitative  information, 
o     Frank  stated  Qualitative  data  addresses  emerging  groups  as  well  as  hidden  issues. 

In  way  of  example  he  noted  that  the  "Down  lovJ'  population  has  been  known  to  exist 

for  a  long  time,  is  very  difficult  to  study,  and  is  under-served. 
^>     Kym  stated  that  a  good  description  of  what  is  going  or\  in  any  community  needs  to  be 
more  than  numbers,  the  anecdotal  information  adds  emotion  and  perspective. 

She  cited  the  photography/essay  exhibition  (called  phototalk)  at  City  Hall  that  the 

Prop  10  Coalition  put  together  to  explain  the  impact  of  the  tobacco  tax  on  funding 

as  being  both  powerful  and  moving. 
■=>     Frank  noted  that  some  things  can  not  be  adequately  reported  digitally,  or  by  numbers, 

and  that  Qualitative  data  fills  in  the  inadequacies  of  ELI  and  PEAAS. 
■=>    Mike  stated  that  the  challenge  is  building  into  the  Quantitative  data  required  by 
funders  sufficient  Qualitative  information  to  provide  a  whole  picture. 

6.    Brainstorm  Ideas  for  Strategic  Plan 

•  Kym  turned  the  group's  attention  to  page  2  in  the  document,  suggesting  the  next  step  might 
be  for  the  committee  to  ask  new  questions,  such  as  those  listed  on  the  second  page  of  the 
document. 

•  Dara  reminded  the  committee  of  its  plan  to  develop  a  multi-year  strategic  plan  for  the 
current  levels  of  prevention  activity:  Providers;  City-Wide;  and  Intervention  Research. 

■=>     She  added  that  the  Committee  should  concentrate  this  year  on  the  previously  agreed 
three  priority  evaluation  areas:  Quality  Assurance;  Referral  Tracking,  and  Cost 
Effectiveness/Program  Assessment. 

Perry  noted  that  clarification  was  needed  as  to  how  the  three  areas  fit  into  a  multi-year 

plan. 

■=>     Dara  explained  that  the  previous  plan  of  action  was  based  the  next  RFP  being  out  in 
May,  but  the  AIDS  Office  now  anticipates  the  RFP  at  the  end  of  the  calendar  year. 

•  Mike  pointed  out  that  this  process  is  ar\  opportunity  to  come  up  with  a  new,  innovative,  and 
cutting  edge  way  of  looking  at  evaluation. 

Perry  noted  that  evaluation  is  important  on  several  levels,  including  that  many  members  also 

have  responsibility  as  program  managers. 

■=>     Dara  stated  that  the  providers'  viewpoint  should  be  considered,  but  that  the  Committee 

needs  to  take  a  planning  perspective,  to  develop  a  plan  for  the  City  as  a  whole. 
o    Ari  suggested  finding  ways  to  collect  data  on  all  levels  and  from  all  perspectives;  but 

that  members  be  clear  to  keep  the  viewpoints  separate, 
o    Tracey  stated  that  the  Committee's  prime  objective  is  planning. 
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She  added  that  the  focus  should  be  on  what  best  informs  the  planning  process  for 

prevention. 
Frank  suggested  that  since  there  may  be  time  to  do  so,  it  would  be  very  valuable  to  find  out 
from  the  current  stakeholders  what  has  been  effective. 

<=>     He  added  that  they  might  have  a  different  view  of  evaluation  than  the  AIDS  Office. 
=>    He  pointed  out  that  service  providers  often  hear  evaluation  as  criticism. 
Mike  suggested  the  Committee  take  a  strategic  view  of  evaluation. 
■=>     Kym  added  that  she  thinks  of  the  Strategic  Planning  process  as  looking  at  the 
timeframe  and  what  participants  want  to  get  out  of  it. 

She  suggested  looking  at  the  big  picture,  including  what's  happening  elsewhere  in  the 

U.S.  and  around  the  world. 
Frank  pointed  out  that  there  has  been  a  history  of  contradictory  and  confusing  guidance 
from  the  DPH,  for  instance:  agencies  were  told  not  to  design  programs  around  Units  of 
Service  and  then  later  asked  to  evaluate  their  programs  based  on  Units  of  Service. 

•  Dara  stated  that  a  strategic  approach  is  about  the  different  levels  of  data.  It  is  not  about 
adding  to  providers'  data  collection  burden. 

"=>     She  added  that  some  of  what  is  already  being  collected  might  not  need  to  be  continued. 
Mike  suggested  a  presentation  to  the  Committee  about  the  Strategic  Planning  process 
before  going  to  the  Council  or  community  -  including  the  cutting  edge  stuff  being  done. 
■=>    He  added  that  it  could  be  put  together  by  the  Committee's  next  meeting  04/01/04. 
Perry  emphasized  that  the  Committee  needs  to  keep  in  mind  how  the  process  gets 
evaluation  information  back  to  the  front-line  service  provider. 

<=>     He  added  that  the  process  might  also  assess  how  they  react  to  the  evaluation  data. 
<=t>    Mike  added  that  it  would  be  valuable  to  gain  a  Qualitative  view  by  asking  front-line  staff 

what  the  data  means  to  them. 
As  an  example  of  merging  Quantitative  and  Qualitative  approaches,  Frank  cited  Jen 
Sarche's  presentation  on  vaccines  which  gives  data  and  explains  the  meaning  of  the  data. 
Dara  suggested  that  focus  groups  are  a  way  to  get  a  Qualitative  view: 

Asking  providers  how  the  process  is  working  for  them,  and 

Gathering  clients  and  asking  them  what  services  are  missing. 
<=>    Mike  added  that  such  a  review  should  include  talking  with  people  not  receiving  service. 

•  Frank  noted  that  the  second  part  of  Dara's  suggestion  is  being  worked  on  by  a  number  of 
agencies  associated  with  the  DPH  and  suggested  looking  into  what  they  are  finding. 

■=>    Mike  suggested  assembling  a  list  of  Qualitative  Evaluation  efforts  underway  in  SF. 
■=>    Marcel  agreed  that  the  Committee  should  inform  itself  about  the  data  being  collected, 

particularly  what  is  being  learned  about  client  satisfaction. 
■=>    Mike  added  that  it  would  also  be  useful  to  find  out  what  other  jurisdictions  are  doing. 
Frank  cautioned  that  conducting  Focus  Groups  with  providers,  clients,  and  potential  clients 
would  set  up  the  expectation  that  something  will  happen  as  a  result. 

Summing  Up 

Tracey  suggested  summing  up  the  brainstorming  for  the  Strategic  Plan. 

Dara  asked  if  the  Committee  has  agreed  to  do  Strategic  Planning  regarding  evaluation. 
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■=>    Perry  asked  if  there  were  any  dissenting  opinions,  or  other  comments.   None  were 

offered.   Agreement  to  do  Strategic  Planning  regarding  evaluation  was  by  consensus. 
Perry  asked  if  the  committee  agreed  to  examine  other  evaluation  options,  approaches,  and 
data. 

o    Tracey  stated  that  Qualitative  data  could  be  of  great  value  to  the  committee's  planning 
process,  and  that  it  should  include  looking  at: 

What  service  providers  get,  and  would  like  to  get,  from  the  evaluation  process, 
What  other  jurisdictions  are  doing,  and 

What  clients  and  those  not  receiving  services  are  getting  and/or  missing  from 
present  services. 
■=>    Perry  asked  if  any  members  had  further  comments. 
■=>     Frank  added  that  the  guiding  principle  should  be  incorporating  effective,  accurate 

measures  into  the  Evaluation  process  including  Qualitative  data. 
o    Marcel  added  that  the  strategic  plan  should  feed  into  the  planning  process. 
Consensus  was  reached  to  examine  other  evaluation  as  summarized  by  Tracey  and  Frank. 
.      Kym  asked  if  it  would  be  helpful  for  her  to  map  out  a  strategic  planning  process. 
There  was  general  agreement  that  this  would  be  helpful. 

o     Frank  stated  that  he  would  like  to  see  a  reading  packet  on  the  strategic  planning 
process  for  next  meeting  04/01/04. 

Summary,  Evaluation  and  Closure 
Evaluation  Process 

Tracey  distributed  a  draft  evaluation  form  entitled,  "Evaluation  Committee  Meeting  Evaluation' 
a  copy  of  which  is  available  to  absent  members  upon  request.   She  explained  that  she  has 
reviewed  the  agreed  upon  procedures  as  well  as  the  draft  evaluation  form  with  Kevin  Roe  of  the 
Process  Evaluation  Team.   She  asked  the  committee  to  review  the  forms. 

Frank  stated  that  he  liked  using  the  form,  although  he  doesn't  like  forms. 
.      Perry  asked  if  there  was  agreement  to  using  the  form  as  drafted.   No  dissenting  opinions 

were  offered  and  there  was  no  further  discussion.  The  evaluation  form  was  accepted  by 

consensus. 

Evaluation  forms  were  distributed  for  use.  The  meeting  adjourned  4:57  PM. 

THE  NEXT  MEETING  IS  SCHEDULED  FOR  THURSDAY,  APRIL  1,  2004 
FROM  3:00  TO  5:00  PM  -  ROOM  330B 


Minutes  were  prepared  by  David  Weinman,  Reviewed  by  Tracey  Packer,  and  Perry  Rhodes  III. 
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<r  Evaluation  Committee 

Thursday,  April  1,  2004 

3:00  pm  -  5:00  pm 

25  Van  Ness  Avenue,  330B 

^San  Francisco 


^AGENDA 

1.  Welcome  and  Announcements  3:00-3:10 

2.  Public  Comment  3:10-3:20 

3.  Approval  of  Minutes  from  March  4,  2004  3:20-3:25 

4.  Report  from  Steering  Committee  3.25-3:35 

5.  Review  Steps  of  Strategic  Plan  and  Worksheet  (possible  vote)  3:35-3:45 

6.  Begin  Process  of  Developing  Strategic  Plan:   Develop  6oals  (vote)  3:45-4:15 

7.  Categorize  Committee  Priorities  within  Goals  4:15-4:25 

8.  Discuss  First  Action  Step:  6LI  and  PEMS  (possible  vote)  4:25-4:50 

9.  Next  Steps,  Evaluation,  and  Closure  4:50-5:00 

Next  meeting  will  be  May  6,  2004  at  3PM 


NOTE:  All  meetings  are  open  to  the  public  and  are  held  in  handicapped  accessible  facilities. 
Meeting  dates  and  times  are  subject  to  change,  please  verify  by  calling  Betty  Chan  Lew  at  554-9492. 

Know  your  rights  under  the  Sunshine  Ordinance:  Government's  duty  is  to  serve  the  public,  reaching  Its  decisions  in  full  view  of  the 
public.  Commissions,  boards,  councils,  and  other  agencies  of  the  city  and  county  exist  to  conduct  the  people's  business.  This 
ordinance  assures  that  deliberations  are  conducted  before  the  people  and  that  City  operations  are  open  lo  the  people's  review.  For 
more  information  on  your  rights  under  the  sunshine  ordinance  or  to  report  a  violation  of  the  ordinance,  contact  the  Sunshine 
Ordinance  Task  Force.   Donna  Hall,  City  Hall,  Room  244,  1  Dr.  Carlton  B.  Goodiett  Place,  San  Francisco,  CA  94102,  Phone;  554- 
7724,  Fax:  554-7854,  E-Mail:  Donna_Hall@d$f-Ca.us. 

In  order  to  assist  with  the  City's  efforts  to  accommodate  persons  with  severe  allergies,  environmental  illnesses,  multiple  chemical 
sensitivity  or  related  disabilities,  attendees  at  public  meetings  are  reminded  that  other  attendees  may  be  sensitive  to  various 
chemical  based  products.  Please  help  the  City  accommodate  these  individuals. 
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Members  Present:  Chata  Ashley,  Ari  Bachrach,  Lisa  Reyes  (HPS),  Perry  Rhodes  III,  Israel  Nieves- 
Rivera  and  Frank  Strona. 

Members  Absent:  Michael  Discepola 

Professional  Staff:   Kym  Dorman  (Harder  &  Co),  Tracey  Packer  (HPS),  and  Mike  Pendo  (HPS) 

1.  Welcome  and  Announcements 

Perry  Rhodes  III  called  the  meeting  to  order  at  3:15  PM.   He  invited  attendees  to  introduce 
themselves  and  make  relevant  announcements. 

Michael  D.  sent  an  email  stating  that  he  would  not  be  able  to  attend  today's  meeting 

2.  Public  Comment 

There  was  no  public  comment. 

3.  Approval  of  Minutes  from  March  4th,  2004 

Ari  mentioned  that  on  page  4,  2nd  bullet  of  the  3/4/04  minutes  that  the  essence  of  what  he 
said  was  not  captured  correctly.  Ari's  comment  will  be  changed  to  reflect  his  comment  about 
cost  effectiveness  and  not  include  about  gay  white  men  during  this  conversation. 

Motion  was  made  to  approve  the  minutes,  with  the  above  changes.   There  was  no  further 
discussion,  or  objections.  The  minutes  were  approved  by  consensus. 

An  also  mentioned,  that  for  the  record,  he  uses  the  pronoun  "he." 

4.  Report  from  Steering  Committee 

Perry  provided  an  overview  of  the  latest  Steering  Committee  meeting.   He  mentioned  the  issue 
of  having  outside  presenters  at  council  meetings.  Specifically,  he  was  referring  to  the  February 
HPPC  meeting,  where  a  presenter  made  a  comment  that  some  council  members  found  offensive. 
It  was  suggested  that  in  the  future,  when  instances  like  this  occur,  that  if  members  don't  feel 
comfortable  bringing  it  up  at  that  moment,  that  they  can  pass  a  note  to  co-chairs,  or  they  can 
bring  it  up  in  process  evaluation.  Perry  also  reported  that  due  to  budget  changes,  evaluation  will 
be  done  through  Zoomerang  surveys,  rather  than  paper  surveys  at  all  committees. 

5.  Review  Steps  of  Strategic  Plan  and  Worksheet,  6.  Begin  Process  of  Developing  a  Strategic 

Plan;  and  7.    Categorize  Committee  Priorities  within  Goals. 

Kym  distributed  a  revised  draft  plan  and  worksheet  for  the  2004-2008  strategic  plan. 
Perry  and  Kym  walked  committee  members  through  the  8  page  document. 

After  much  discussion  on  evaluation  and  the  2004-2008  strategic  plan,  it  was  decided  that 
Frank,  Perry  and  Kym  will  meet  to  develop  goals  based  on  the  vision  statements  from  pages  2-4. 
These  goals  will  then  be  submitted  to  another  small  group  of  the  committee,  who  will  then  work 
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on  action  steps  for  each  (Israel,  Lisa,  Ah,  Mike,  Tracey,  and  Kym).   Both  sets  of  ideas  will  be 
brought  to  the  May  6  committee  meeting.  The  committee  agreed  to  this  by  consensus. 

The  discussion  included  the  following  comments: 

Evaluation  is  a  process  by  which  funded  agencies  can  improve  programs  and  better  meet 

needs  of  target  populations. 

Evaluation  is  a  planning  tool. 

Include  how  the  monitoring  report  affects  evaluation.  Monitors  should  be  advisors  rather 

than  overseers. 

Measure:  were  you  able  to  complete  what  you  contracted  to  do. 

Include  more  about  how  the  agency  can  reflect  upon  itself  and  wonder  if  what  they  did  was 

what  they  should  have  done. 

Ability  to  change  programs  based  on  your  experience. 

How  do  we  acknowledge  the  need  to  change  programs  based  on  funding? 

Develop  plan  in  steps-what  can  be  accomplished  each  year. 

How  do  we  ensure  that  community  providers  agree  with  this  vision  and/or  can  buy  into  the 

vision? 

Make  evaluation  relative  to  the  larger  scope  of  the  work  that  providers  are  attempting- 

change  in  broader  life  issues. 

Some  people  think  the  Plan  isn't  enforced. 

Evaluation  gives  voice  to  consumers  of  HIV  prevention. 

Referrals  allow  program  to  meet  needs  beyond  prevention  needs. 

8.  Discuss  1st  Action  Step:    ELI  and  PEMS 

Perry  explained  that  this  discussion  had  been  referred  to  the  Evaluation  Committee  from 
HPPC.  The  goal  of  this  agenda  item  is  to  discuss  how  ELI  and  PEMS  fit  into  the  strategic 
plan  but  also  to  acknowledge  some  of  the  difficulties  caused  by  ELI.  One  member  asked 
how  we  can  have  a  discussion  on  PEMS  when  it  hasn't  been  developed  yet.    Tracey 
mentioned  that  council  members  are  asking  for  information  on  ELI/PEMS  and  are  asking  for 
guidance  around  the  issue.  Israel  mentioned  that  ELI  was  not  intended  to  be  used  for 
reimbursement,  yet  it  is  being  used  that  way.  Frank  suggested  that  we  ask  start  by  asking, 
"What  data  is  useful?"  Members  agreed.  It  was  suggested  that  we  ask  the  question  "What 
data  is  useful?"  at  this  committee,  then  to  HPPC  members,  then  get  community  input,  which 
includes  providers.   And,  it  was  suggested  that  the  results  should  be  sent  to  CDC/PEMS 
development  folks.  Members  all  agreed  on  this. 

9.  Next  Steps,  Evaluation,  and  Closure 

Evaluation  forms  were  distributed  for  use.  The  meeting  adjourned  5:00  PM. 

THE  NEXT  MEETING  IS  SCHEDULES  FOR  THURSDAY,  MAY  6,  2004 
FROM  3:00  TO  5:00  PM 

Minutes  were  prepared  by  Lisa  Reyes  and  Tracey  Packer,  reviewed  by  Perry  Rhodes  III. 
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1.  Welcome  and /Announcements  3:00-3:10 

2.  Public  Comment  3:10-3:20 

3.  Approval  of  Minutes  from  April  1,  2004  3:20-3:25 

4.  Report  from  Steering  Committee  3:25-3:35 

5.  Review  Draft  of  Strategic  Plan  and  Worksheet  (possible  vote)  3:35-3:55 

6.  Continue  Development  of  Strategic  Plan:  Develop  Action  Steps         3:55-4:30 
(possible  vote) 


7.  Discuss  ELI  and  PEMS  (possible  vote) 

8.  Next  Steps,  Evaluation,  and  Closure 


4:30-4:50 
4:50-5:00 


Next  meeting  will  be  June  3,  2004  at  3PM 


NOTE:  All  meetings  are  open  to  the  public  and  are  held  in  handicapped  accessible  facilities. 
Meeting  dates  and  times  are  subject  to  change,  please  verify  by  calling  Betty  Chan  Lew  at  554-9492 

Know  your  rights  under  the  Sunshine  Ordinance:  Government's  duty  is  to  serve  the  public,  reaching  its  decisions  in  full  view  of  the 
public    Commissions,  boards,  councils,  and  other  agencies  of  the  city  and  county  exist  to  conduct  the  people  s  business    This 
ordinance  assures  that  deliberations  are  conducted  before  the  people  and  that  City  operations  are  open  to  the  people's  review    For 
more  information  on  your  rights  under  the  sunshine  ordinance  or  to  report  a  violation  of  the  ordinance,  contact  the  Sunshine 
Ordinance  Task  Force.  Donna  Hall,  City  Hall.  Room  244,  1  Dr  Carlton  B.  Goodlett  Place.  San  Francisco.  CA  94102.  Phone:  554- 
7724.  Fax    554-7854.  E-Mail:  Donna_Hall@ci.sf  ca  us 

In  order  to  assist  with  the  City's  efforts  to  accommodate  persons  with  severe  allergies,  environmental  illnesses,  multiple  chemical 
sensitivity  or  related  disabilities,  attendees  at  public  meetings  are  reminded  that  other  attendees  may  be  sensitive  to  various 
chemical  based  products    Please  help  the  City  accommodate  these  individuals 
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Members  Present:  Marcel  Miranda,  Lisa  Reyes  (HPS),  Perry  Rhodes,  III,  and  Frank  Strona. 

Members  Absent:   Chata  Ashley,  Michael  Discepola,  and  Israel  Nieves-Rivera. 

Professional  Staff:  Kym  Dorman  (Harder  A  Co),  Mike  Pendo  (AIDS  Office),  and  David  Weinman 
(Note  Taker). 

1.  Welcome  and  Announcements 

Perry  Rhodes,  III  called  the  meeting  to  order  at  3:12  PM.  He  invited  attendees  to  introduce 
themselves  and  make  relevant  announcements. 

Lisa  Reyes  announced  that  the  first  Plan  Training  would  take  place  on  Tuesday,  05/11/04 

from  1:00  to  4:00  PM.   Light  refreshments  will  be  served.   She  noted  that  the  second 

training  is  scheduled  for  05/25/04. 

Frank  Strona  announced  that  Safe  Sex  City's  first  social  event  is  scheduled  for  07/03/04. 

2.  Public  Comment 

None 

3.  Approval  of  Minutes  from  April  1,  2004 

Acceptance  of  the  minutes  was  initially  postponed  until  the  06/03/04  meeting.  Later  in  the 
meeting,  when  quorum  was  present,  motion  was  made  and  seconded  to  accept  the  minutes. 
There  was  no  discussion,  objection,  or  abstention.  The  minutes  were  approved. 

4.  Report  from  Steering  Committee 

Frank  reported  developments  from  the  Steering  Committee,  including  the  following. 
o    Lisa  announced  that  an  HPS  abstract  titled:  "Ensuring  Parity,  Inclusion  and 

Representation  in  Your  CPG:   San  Francisco's  HIV  Prevention  Planning  Council"  was 
accepted  at  the  2004  HIV  Prevention  Leadership  Conference  in  Atlanta.   She  will  be 
presenting  with  Gwen  Smith.  It  was  also  mentioned  that  Israel  Nieves-Rivera  was  asked 
to  speak  on  "10  Years  of  Community  Planning"  at  the  plenary  session. 

Jason  Riggs  from  the  STOP  AIDS  Project  came  to  Steering  to  speak  on  CDC 
Directly-funded  agencies. 

5.  Review  Draft  of  Strategic  Plan  and  Worksheet 

Kym  Dorman  noted  that  the  document  entitled,  "Changing  the  Culture  of  Evaluation  in  5F  HIV 
Prevention"  had  been  e-mailed  to  all  members.   The  Goals  were  developed  in  a  subcommittee  of 
her,  Frank,  and  Perry.   She  linked  the  Objectives  to  the  Goals.   Discussion  followed. 


Draft  Introduction 


Frank  said  that  the  Introduction  is  proposed  as  a  narrative  of  issues  needing  to  be 
addressed,  including  those  areas  of  concern  that  don't  rise  to  level  of  the  stated  Goals. 
He  also  explained  that  the  Introduction  is  the  way  of  framing  the  topic. 
<=>     Evaluation  is  not,  he  emphasized,  a  four-letter  word. 
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Kym  pointed  out  that  the  process  is  part  of  a  way  of  looking  at  evaluation  differently,  as 
empowerment,  and  a  means  of  helping  the  prevention  community  get  where  it  wants  to  go. 
Kym  explained  the  plan's  overall  structure: 
<>     Strategies  are  the  various  approaches  to  the  Goal's  issues, 

^  Objectives  are  the  operational  means  by  which  the  Strategies  will  dealt  with,  and 
=*  The  Action  Steps  are  the  specific  activities  needed  to  accomplish  the  Objectives. 
She  provided  a  brief  overview  of  the  three  Goals. 

■'<     She  observed  that  Goal  2  is  a  review  of  evaluation  strategies  in  use,  and  provides  a 
feedback  loop. 

Frank  pointed  out  that  Goal  2  is  a  "Matrix!1  laying  out  who  is  responsible,  as  well  as 
who  is  doing  what  -  as  discussed  at  the  last  Committee  meeting  (04/01/04). 
Kym  explained  that  the  subcommittee  discussed  the  interaction  between  the  three  Goals. 
<=>     Frank  noted  the  interdependent  relationship  between  the  Goals. 
■=>    He  highlighted  that  it  isn't  a  matter  of  achieving  the  three  goals  and  then  walking  away. 

Achieving  each  Goal  is  dependent  on  the  progression  of  the  others. 
■=>    Mike  Pendo  observed  that  the  relationship  between  the  goals  may  not  necessarily  be 

cyclical,  but  dynamic. 
^    He  also  stated  that  the  community  is  referred  to,  but  is  not  explicit  in  the  Goals. 
Discussion  followed  about  how  to  describe  the  Goals'  relationship/interdependencies.  The 
following  diagrams  the  Committee's  agreed  overview,  with  the  understanding  that  details  are  to 
be  settled  later. 


Continue  Development  of  Strategic  Plan:    Develop  Action  Steps 

The  Committee  then  discussed  procedures  and  how  to  proceed,  including  the  following: 

Marcel  Miranda  stated  that  documents  need  to  be  sent  to  members  further  in  advance  than 

the  day  before  the  meeting. 

Perry  suggested  reviewing  each  Goal  and  Objective  in  order  presented. 

Mike  suggested  providers  be  given  options  on  how  to  do  things,  perhaps  providing  examples. 

■=>    Mike  said  that  the  process  should  encourage  creative  approaches. 

<=>    Frank  said  that  examples  imply  a  preferred  approach. 
The  Committee  agreed  to  brainstorm  Goals  and  Strategies. 

(Listed  here  sequentially,  although  not  necessarily  discussed  in  that  order.) 


Goal  1:  Evaluation  reflects  how  SF  HIV  prevention  efforts  promote  the  mental,  physical, 
emotional  and  structural  health  and  well-being  of  people  in  SF. 


Goal  1  -  Strategy  1:  (Research  Interventions) 

It  was  agreed  that  the  Evaluation  Committee  should  prioritize  the  issues  researched. 
Goal  1  -  Strategy  2   (Track  CDC  Performance  Indicators) 
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No  discussion,  additions  or  changes  were  offered. 

Goal  1  -  Strategy  3:  (Research  on  Impact  of  HIV  Prevention...  on  Health  and  Well-being.) 

<=>    Mike  stated  that  the  term  "impact'  is  used  more  often  when  referring  to  quantitative 

data  rather  than  qualitative. 

He  added  that  some  people  dismiss  any  data  categorized  as  qualitative. 
■=>     Kym  expressed  the  value  and  validity  of  qualitative  data. 
■=>     Frank  and  Lisa  suggested  words  like,  "effect." 
Goal  1  -  Strategy  4  (Referral  Tracking  System) 
■=>     Frank  suggested  that  any  system  developed  be  coordinated  with  work  in  progress  by  the 

CARE  Council  on  an  integrated  service  model. 
<=>     Kym  suggested  using  the  term,  "Build  rather  than,  "Develop. " 
^>     Mike  stated  this  Committee  may  want  to  consider  making  recommendations  about 

development  of  a  system,  rather  than  actually  developing  it. 

He  added  using  a  model  similar  to  the  Case  Management  recommendations. 
<=>    Perry  suggested  changing  the  wording  to  something  like,  "Make  recommendations  about 

the  development  of  a  Referral  Tracking  system...  " 
o     Kym  suggested  the  Committee  "coordinate  information,"  rather  than  "develop," 

"implement,"  or,  "identify  tracking  steps." 
■=>    Marcel  stated  that  developing  a  Referral  Tracking  system  is  too  big  a  commitment. 
>=>    He  and  Perry  suggested  the  Committee  facilitate  discussions;  perhaps  host  community 

meetings  and/or  provide  a  sense  of  what  good  tracking  looks  like. 
^    Mike  noted  that  the  SF  DPH's  Behavioral  Department  is  also  working  on  linkage  and  that 

their  input  could  be  valuable, 
o    There  was  general  agreement  to  the  suggestions  that  the  Committee  develop 

recommendations  through  community  input  for  a  Referral  Tracking  system. 
Goal  1  -  Strategy  5:  (Support  stakeholder  capacity.) 
■=>    Marcel  said  the  term,  "Stakeholder  capacity  is  too  broad,  and  unclear. 

He  advocated  using  simpler  language,  stating  that  a  discussion  about  housing  is 
easier  for  providers  to  understand  than  one  about  structural  health. 
■=>    Mike  said  that  this  Strategy  should  focus  on  capturing  data. 
o    Lisa  suggested  rewording  this  to  "Provide  technical  support  to  agencies  on  data 

collection  requirements." 
■=>     Kym  stated  that  it  is  also  about  what  providers  can  do  relevant  to  Goal  1. 
■=>    Marcel  suggested  the  wording  be  more  concrete,  such  as:  "...  contract  monitoring, 

technical  assistance,  and  training." 

Frank  suggested  adding  the  concrete  example  of  "invoicing  training." 
■=>     Mike  suggested  ensuring  that  the  data  collected  component  of  contract  requirements 

also  be  useful  and  valuable  to  providers  and  their  clients, 
o    Lisa  suggested  the  Strategy  cover  two  issues:  1)  Ensure  everyone  is  speaking  same 

language,  2)  Provides  agencies  with  technical  assistance. 

Kym  said  that  technical  assistance  should  ensure  data  is  collected  and  evaluation  is 

executed  accurately. 

Marcel  added  that  providers  need  to  understand  what  is  expected  of  them. 
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"=>     Kym  suggested  the  Committee  create  a  guide  clearly  defining  the  Goals. 
"=>    Mike  pointed  out  that  this  Strategy  also  refers  to  Goal  3  -  Strategy  4  (...  how  to  create 
and  support  systems  that  will  implement  and  sustain  the  new  prevention  efforts.) 

Both  Strategies  (Goal  1-5  and  Goal  3-4)  should  ensure  evaluations  are  useful  for 

providers,  clients,  and  the  community. 
There  was  general  agreement  with  the  suggested  amendments  to  this  Strategy. 
Goal  1  -  Strategy  6:  (Complete  Client  Satisfaction  Surveys) 
o     Frank  questioned  the  use  of  the  word,  "Complete!'  regarding  client  satisfaction  surveys. 

Kym  explained  that  it  was  used  in  the  sense  of  "Execute",  or  "Fulfill' 
o    Lisa  asked  about  the  use  of  the  word,  "enjoyable"  in  the  Action  Step. 

Perry  explained  it  as  an  attempt  at  lightening  the  process  and  thereby  encouraging 

participation. 

Frank  suggested  using  "meaningful'  rather  than  "enjoyable." 
■=>    Mike  suggested  opening  up  the  survey  and  allowing  clients  to  have  input  into  the 
processes;  and  in  so  doing  improving  the  prevention.  (Continuous  quality  improvement 
model.) 
o    Marcel  and  Kym  suggested  replacing  "Complete..."  with  "Fulfill..." 
Goal  1  -  Strategy  7:  (Fulfill  Required  Outcome  Objective) 

o     Standardized  Outcome  Objectives  -  there  was  discussion  regarding  the  possibility  of 
the  HPPC,  the  CDC  (or  others)  developing  standardized  objectives,  including: 

Mike  noted  that  Outcome  Objectives  could  be  standardized,  or  program  specific. 

*  He  questioned  if  the  HPPC  is  moving  toward  standardized  outcomes. 
Frank  said  that  data  on  program  participation  could  be  standardized  and  is 
necessary  for  any  evaluation  of  outcomes. 

*  He  highlighted,  however,  that  some  outcomes  are  population  specific. 
Mike  said  that  it  seems  the  CDC  is  moving  in  the  direction  of  standardizing. 

*  He  urged  the  Committee  to  look  at  standardization,  so  that  the  HPPC  can  help 
define  and  shape  any  required/standardized  Outcome  Objectives  required  by 
the  CDC  or  other  f  unders. 

Perry  said  that  he  would  support  the  HPPC  playing  an  active  role  in  shaping  any 

standardized  Outcome  Objectives  that  may  be  formulated. 

Marcel  and  Mike  suggested  the  wording, "  The  Evaluation  Committee  will  participate 

in  the  development  and  implementation  of  standardized  Performance  and  Outcome 

Objectives" 

*  They  added  that  Committee  members  should  be  included  in  the  DHP  workgroup, 
if  formed 

There  was  general  agreement  to  this  approach. 
<=>    Other  Aspects  of  Strategy  7  were  discussed,  including: 

Marcel  said  that  the  term  "fulfill'  should  be  dearer. 

Kym  suggested  the  approach  of  fulfilling  mandated  Outcomes  while  at  the  same  time 

attempting  to  make  the  requirements  relevant  to  the  needs  of  the  SF  communities. 

Marcel  suggested  referring  to  the  2004  Plan  by  footnote  in  the  Strategic  Plan. 
Goal  1  Strategy  8:  (Data  Collection  requirements  for  ELI/PEMS) 
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<=>     Frank  suggested  changing  " Fulfill'  to  " Support  providers  in  fulfilling  and  utilizing  ..." 

■=>    Marcel  noted  that  the  PEMS  system  doesn't  actually  exist  as  yet,  and  suggested  waiting 

until  it  does  and  if  it's  use  us  decided  upon. 
^>    Mike  noted  that  this  Committee  doesn't  have  much  voice  in  whether  PEMS  or  ELI  will  be 
used,  but  whichever  is  chosen,  the  committee  car\  impact  the  system's  usefulness  and 
relevance. 

The  Committee  may  also  ensure  a  smooth  transition  from  the  present  to  the  future 

system. 
o    Perry  stated  the  he  would  not  be  comfortable  deciding  between  ELI  and  PEMS. 
^>     Kym  suggested  deleting  the  Action  Step,  "Eval  Committee  discuss  ELI/PEMS..."  as  it 

doesn't  have  real  input  into  the  choice  process. 
<=>    Mike  suggested  replacing,  "ELI/PEMS'  with  "database  collection  tools." 
There  was  agreement  on'. 

Action  Step  1  -  ensuring  whatever  system  is  implemented  is  useful,  and 

Action  Step  2  -  making  sure  that  the  Goals  and  Objectives  are  supported  by  the 

data  collected  in  the  system. 
Goal  1  -  General: 
<=>     Mike  asked  about  dissemination  of  findings  and  wondered  if  it  should  be  included  Goal  1. 

Kym  noted  that  this  is  dealt  with  in  Goals  2  and  3. 

Lisa  noted  that  it  is  also  covered  in  the  2004  Plan. 

Kym  will  add  specific  language  on  dissemination  of  information  to  Goal  2. 
<=>    Marcel  made  several  recommendations  regarding  the  Action  Steps,  including: 

Clarity  as  to  who  is  responsible  for  each  of  the  steps; 

The  need  for  community-wide  efforts,  rather  than  the  Committee  telling  people 

what  to  do;  and 

That  they  should  be  easier  for  providers  to  understand  and  work  with. 

*  He  noted  that  for  years  ease  of  use  and  understanding  has  been  discussed  in 
relation  to  monitoring  reports. 

■=>     Kym  stated  that  gathered  from  the  discussion  of  these  Strategies,  that  the  members 
favor  making  the  CDC's  requirements  more  useful  and  valuable  to  providers  and  clients. 
She  added  that  the  this  has  three  phases: 

*  Short-Term  -  fulfill  those  requirements  mandated  by  the  CDC; 

*  Medium- Term  -  gather  other  useful  information  to  share  with  providers  for 

program  improvement;  and 

*  Long-Term  -  engage  with  providers  to  change  the  measurement  tools. 
■=>    This  strategic  plan,  she  noted,  is  about  changing  what  has  to  be  done  into  what 

providers  want  to  do  to  improve  their  processes. 

Marcel  said  that  another  purpose  is  show  providers  that  evaluation  can  be  helpful 
rather  than  a  burden. 
<=>     Lisa  suggested  that  this  committee  would  benefit  from  inviting  a  Program  Manager  from 

the  AIDS  Office  to  future  committee  meetings, 
o    Mike  said  that  the  process  of  transforming  the  requirements  into  a  valuable  process  is  a 
learning  process  for  the  providers  and  the  AIDS  office. 
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Motion  was  made  and  seconded  to  accept  Goal  1  including  the  Strategies  and  Action  Steps 
discussed.   Discussion  followed. 

Kym  stated  that  the  motion  should  have  a  provision  that  approval  is  preliminary,  and  that 

final  approval  will  be  made  after  review  of  her  rewrite. 

Mike  added  that  Goal  l's  final  approval  should  accommodate  amendments  that  may  arise  as 

the  Committee  works  on  Goals  2  and  3. 

Marcel  suggested  a  brief  meeting  to  finalize  approval  before  the  Council  meeting  05/13/04. 

o     Frank  stated  that  the  Committee  doesn't  need  another  meeting. 

o     Frank  and  Kym  suggested: 

The  rewrite  be  sent  to  all  members 

At  the  next  Committee  meeting  (06/03/04)  final  approval  be  decided,  and 

The  focus  of  the  06/03/04  meeting  be  to  review  of  Goals  2  and  3. 

o    Kym  stated  that  she  would  e-mail  the  rewrite  to  all  members  by  Tuesday  05/11/04. 
David  Weinman  said  that  he  would  get  his  notes  to  Kym  by  Monday  (05/10/04). 

o    Perry  will  also  send  a  message  to  all  members  to  expect  something  by  05/11/04. 

Marcel  suggested  each  member  in  attendance  contact  ar\  absent  member  to  review  what  has 

been  discussed. 

o     He  offered  to  talk  with  Michael  Discepola. 
There  was  consensus  on  the  motion  as  amended  and  Goal  1  including  its  Strategies  and  Action 
Steps  were  approved. 

Kym  asked  if  the  Committee  to  approve  applying  the  Short,  Medium,  and  Long-Term  phases  (as 
described  above)  to  all  Strategies  involving  mandated  evaluation  processes  (Strategies  2,  5,  6,  7 
and  8).  The  consensus  of  the  Committee  was  to  approve  the  application  of  the  phases  described 
to  the  relevant  Strategies  in  Goal  1. 

7.  Discuss  ELI  and  PEMS 

It  was  agreed  that  this  item  was  referred  to  the  06/03/04  meeting. 

8.  Next  Steps,  Evaluation,  and  Closure 

Perry  stated  that  it  was  not  practical  to  go  through  Goals  2  and  3  at  this  meeting.  He  also 
suggested  the  Committee  use  the  same  process  for  Goals  2  and  3. 

Frank  stated  that  he  liked  the  procedure  of  a  subcommittee  reporting  back  to  the  whole. 
There  was  general  agreement  to  this  suggestion. 

Evaluation  will  be  by  means  of  Zoomerang. 

The  meeting  adjourned  4:45  PM. 
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THE  NEXT  MEETING  IS  SCHEDULED  FOR  THURSDAY,  JUNE  3,  2004 
FROM  3:00  TO  5:00  PM  -  LOCATION  TBD 


Minutes  were  prepared  by  David  Weinman,  Reviewed  by  Lisa  Reyes,  and  Perry  Rhodes  III. 
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25  Van  Ness  A  venue,  33 OB 

San  Francisco 


DOCUMENTS  DEPT. 
MAR  2  9  2006 

SAN  FRANCISCO 
PUBLIC  LIBRARY 


AGENDA 

1.  Welcome  and  Announcements 

2.  Public  Comment 

3.  Approval  of  Minutes  from  Mav  6,  2004 

4.  Report  from  Steering  Committee 

5.  Continue  Development  of  Strategic  Plan  and  Worksheet  (Possible  Vote) 
>     Goal  2 

r     Goal  3 

6.  Next  Steps,  Evaluation,  and  Closure 


3:00-3:10 
3:10-3:20 
3:20-3:25 
3:25-3:35 
3:35-4:50 

4:50-5:00 


Next  meeting  will  be  July  8th,  2004  at  3PM 


NO  IE:  All  meetings  are  open  to  the  public  and  are  held  in  handicapped  accessible  facilities. 
Meeting  dates  and  times  are  subject  to  change,  please  verify  bv  calling  Bettv  Chan  Lew  at  554-9492. 

Know  vour  rights  under  the  Sunshine  Ordinance:  Government's  duty  is  to  serve  the  public,  reaching  its 
decisions  m  full  view  ot  the  public.    Commissions,  boards,  councils,  and  other  agencies  of  the  city  and 
county  exist  to  conduct  the  people's  business.    This  ordinance  assures  that  deliberations  are  conducted 
before  the  people  and  that  City  operations  are  open  to  the  people's  review.  For  more  information  on  your 
rights  under  the  sunshine  ordinance  or  to  report  a  violation  of  the  ordinance,  contact  the  Sunshine 
Ordinance  Task  Force.   Donna  Flail,  City  Hall.  Room  244,  1  Dr.  Carlton  B.  Goodlctt  Place,  San 
Francisco.  CA    94102,  Phone:  554-7724,  Fax:  554-7854,  E-Mail:  Donna_Hall@ci.sfca.us. 

In  order  to  assist  with  the  City's  efforts  to  accommodate  persons  with  severe  allergies,  environmental 
illnesses,  multiple  chemical  sensitivity  or  related  disabilities,  attendees  at  public  meetings  are  reminded  that 
other  attendees  may  be  sensitive  to  various  chemical  based  products.  Please  help  the  Lit)'  accommodate 
these  individuals. 
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Members  Present:  Lisa  Reyes  (HPS),  Perry  Rhodes,  III,  Israel  Nieves-Rivera,  and  Frank  Strona. 

Members  Absent:   Chata  Ashley,  Michael  Discepola,  and  Marcel  Miranda. 

Professional  Staff:   Kym  Dorman  (Harder  &  Co),  Tracey  Packer  (HPS),  Mike  Pendo  (HPS),  and  David 
Weinman  (Note  Taker). 

1.  Welcome  and  Announcements 

Perry  Rhodes,  III  called  the  meeting  to  order  at  3:11  PM.  He  invited  members  to  make 
announcements. 

Frank  Strona  announced  that  he  is  transitioning  out  of  Walden  House  and  has  accepted  a 
position  with  STD  Prevention  and  Control  of  SFDPH. 
Tracey  Packer  made  announcements  regarding  Committee  members: 
■=>    Chata  Ashley  is  bedridden  with  health  issues  and  will  therefore  miss  some  Council 
meetings. 

He  is  not,  however,  taking  a  leave  of  absence  and  believes  he  will  be  back  soon. 
He  would  love  to  hear  from  members  and  asks  to  remain  nin  the  loop." 
o    Michael  Discepola  has  requested  a  leave  of  absence  from  Council  activities  due  to  his 
workload. 

2.  Public  Comment 

There  was  no  public  comment. 

3.  Approval  of  Minutes  from  May  6,  2004 

Motion  was  made  and  seconded  to  accept  the  minutes  from  the  meeting  05/06/04.  There  was 
no  discussion,  or  objection.  Israel  Nieves-Rivera  abstained.  The  minutes  were  approved 

4.  Report  from  Steering  Committee 

Tracey  Packer  reported  discussion  on  the  following  topics. 

■=>    San  Francisco  has  been  granted  $10-15K  from  for  Rapid  Testing  focused  on  men  of 

color  at  Pride. 
■=>    The  high  absentee  rate  among  members  at  the  last  Council  Meeting  (15/13/4). 
■=>    The  Membership  Committee  is  working  on  identifying  the  gaps  in  Council  membership. 
■=>     A  neighborhood  assessment  (SCAN)  of  Visitacion  Valley  will  be  proposed  to  the  Council. 
■=>     Rafaael  Diaz  will  be  presenting  information  on  the  use  and  effect  of  Methamphetamines 

in  SF  at  the  next  Council  Meeting  (06/10/04). 
■=>    SF  received  $43 K  from  the  State  to  include  Hepatitis  C  with  HIV  programs,  but  the 

programs  to  be  included  have  not  as  yet  been  determined, 
o    No  cut  is  expected  in  City  funding  for  HIV  prevention  services  in  the  upcoming  budget, 
o    The  next  round  of  Requests  For  Proposals  (RFPs)  is  scheduled  for  November  with  due 

date  of  February  2005. 
<=>    The  Membership  Committee  is  also  working  on  Diversity  Training  for 

August/September. 
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Frank  Strona  asked  about  the  process  of  applying  for  Council  membership. 
=>     He  was  told  that  interested  people  should  contact  Betty  Chan  Lew. 

Continue  Development  of  Strategic  Plan  Worksheet 

The  document  entitled,  "Changing  the  Culture  of  Evaluation  in  SF HIV Prevention  2005-2007 
STRA  TEGIC PLAN  FOR  HIV PREVENTION  EVA LUA  TIONINSF'  (the  Strategic  Plan 
Worksheet)  was  distributed.   Kym  Dorman  also  distributed  an  edited/updated  version  with 
additions  highlighted.   Neither  document  was  dated.  Copies  of  both  documents  are  available  to 
absent  members  upon  request.   The  Committee  discussed  the  Strategic  Plan  Worksheet,  as 
follows.  (Please  note,  the  sequence  of  discussions  has  been  organized  by  topic.) 


In  General 


Intended  Audience 

Israel  asked  who  the  intended  audience  is  for  the  Strategic  Plan  Worksheet  -  the  Council, 
providers,  or  others.   Discussion  followed. 

There  was  general  agreement  that  the  audience  for  the  strategic  plan  is  the  same  as  for 
the  Plan,  for  Council,  DPH,  and  providers. 

Who  is  Responsible  for  and  Impacted  by  the  Strategies 

Israel  suggested  adding  a  column  to  the  Strategy  tables  showing  who  is  impacted  by  each 

strategy. 

Mike  and  Tracey  added  that  the  table  should  also  show  who  is  involved  in  each  Strategy 

(who  are  the  players)  -  not  just  who  is  responsible. 
•      Frank  suggested  changing  the  column  title  to, "  Who  is  responsible/involved." 
He  also  raised  the  possibility  of  setting-off  those  involved  in  italics. 

o    Mike  suggested  the  column  be  labeled, "  Who  coordinates /  Who  contributes." 

Israel  and  Perry  suggested,  "Who  is  Responsible:   Coordinator  /  Contributor" 
It  was  agreed  to  keep  the  term  "Responsible,"  but  to  re-draft  the  column  title. 

Strategies  and  Objectives 

Israel  recommended  that  within  each  Goal  the  Strategies  need  to  coordinate  with  the 

Objectives. 

Tracey  suggested  arranging  both  Strategies  and  Objectives  to  coordinate  with  each  other 

and  to  flow  logically. 

Frank  suggested  adding  a  column  to  indicate  which  Objective  is  addressed  by  each 

Strategy. 

Israel  noted  that  the  Strategies  and  Objectives  could/should  be  referenced  in  the  RFPs. 

Evaluation  Committee 

It  was  noted  that  the  Evaluation  Committee  is  not  a  standing  committee  of  the  Council  and 
so  the  Steering  Committee  as  a  standing  committee,  should  Therefore  be  listed  for  on-going 
responsibilities  -  rather  than  the  Evaluation  Committee. 

Overall 
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Tracey  suggested  the  goal  of  the  strategic  plan  is  making  data  required  more  useful  to 
programs,  particularly  so  that  they  can  adjust  and  adapt  their  services  to  the  community's 
needs. 

•  Mike  stated  that  while  data  is  available  and  conveyed  with  service  organizations  he  wonders 
if  it  is  being  used  -  or  if  it  is  being  used  effectively. 

•  Frank  questioned  if  data  is  actually  getting  to  service  providers. 

o     He  cited  recent  Council  meeting  where  members  were  surprised/shocked  by  the 
information  on  the  MTF  and  FTM  communities  although  the  data  is  actually  not  new. 

Perry  stated  that  there  is  an  overall  negative  attitude  regarding  evaluation  processes. 

■=>     He  suggested  addressing  this  by  presenting  data  in  a  format  that  is  friendly  to  the  eye. 

o     He  added  that  getting  providers  to  use  the  available  data  requires  making  it  more 
understandable  as  well  as  meaningful. 

•  Israel  said  that  the  difficulty  is  getting  providers  to  understand  the  importance  of 
available  data. 

•  Perry  suggested  that  the  language  be  similar  in  all  Goals  and  Strategies. 
^>     He  also  suggested  that  it  needs  to  be  exciting. 

•  Kym  noted  Strategy  5  in  Goal  1: 

"Conduct  qualitative  research  on  effect  of  HIV  prevention  efforts  on  health  and 
well-being." 
o     She  suggested  expanding  such  research  to  a  citywide  qualitative  review  in  Goal  2. 
<=>    Tracey  pointed  out  that  it  is  difficult  to  communicate  (Goal  2)  before  data  is  collected 

and  documented  (Goal  1). 
Tracey  observed  that  the  task  before  the  Committee  is  to  make  requirements  more  useful 
to  those  collecting,  reviewing,  and  responding  to  the  data. 
It  was  agreed  that  Kym  would  take  the  members  comments  into  consideration  and  re-draft  the 
Introduction. 


Introduction 


Kim  reviewed  some  of  the  changes  reflected  in  the  edited/updated  version  of  the  Strategic 
Plan  Worksheet,  including  assigning  short  descriptions  of  each  of  the  three  Goals: 
Goal  1  =  Documentation; 
Goal  2  =  Communication;  and 
Goal  3  =  Communication. 
Discussion  followed. 
•      Goal  1  -  use  of  the  term,  "Documentation"  as  descriptor. 

<=>    Lisa  Reyes  noted  the  connotation  of  the  word  documentation  car\  be  negative. 
■=>     Frank  suggested  using  the  term  "Collateral'  instead. 

Kym  said  that  she  wasn't  sure  people  would  know  what "  Collateral' 'means  in  this 
context. 
<=>    Mike  Pendo  said  this  Goal  1  should  tell  the  story  of  how  the  SF  HIV  prevention  efforts 

are  addressing  the  epidemic, 
o    Israel  suggested  using  terms  that  are  more  two-sided,  such  as,  "reflection"  "capture" 
"history"  or,  "sharing"  rather  than,  "document" 
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Goal  2  -  use  of  the  term,  "Communication'  as  descriptor. 

o    Mike  suggested  changing  thumbnail  to,  "Communication/Dissemination." 

<=>    Israel  cautioned  that  "dissemination"  seemed  passive  and/or  one-sided. 

<z>     Frank  suggested  "  Utilization.  " 

Goal  3  -  use  of  the  term,  "Moving  forward'  as  descriptor. 

o     Tracey  suggested  use  of  the  term,  "Innovation.  " 

Other  issues  relating  to  the  Introduction  section. 

^    Mike  suggested  putting  some  explanatory  comment  on  how  "Communications,"  and, 

"Moving  forward'  (Goals  2  &  3)  interact  above  the  two-way  arrow  connecting  them  on 

the  graphic. 
=>     Frank  suggested  adding  some  explanation  of  how  the  Goals  work  together  and  are, 

"dynamic." 


Goal  1  (Documentation) 


Perry  asked  for  comments  about  last  month's  discussion  of  Goal  1  and  its  summary  as 
represented  in  the  Strategic  Plan  Worksheet. 

Frank  complimented  the  work  as  being  comprehensive. 

Israel  asked  about  the  use  of  the  word  "structural' 

<=>    Kym  explained  the  term  as  including  issues  such  as  housing  (see  footnote  ')  and  that  the 
term  is  currently  being  used  in  social  literature  and  documentation. 

^    She  also  stated  that  she  would  send  an  email  to  all  members  providing  more  information 
on  structural  issues. 
The  members  accepted  the  text  of  Goal  1  as  drafted. 

Kym  suggested  adding  CMS  to  Strategy  4  in  Goal  One  ("  Recommend  how  to  enhance 

evaluation  requirements  to  ensure  (they)  reflect  community/program  priorities.") 


Goal  2  (Communication) 


Kym  explained  Goal  2  as  answering  the  question, "  What  are  we  going  to  do  with  the  information 
collected  by  evaluation  mechanisms"  She  added  that  overall  this  goal  can  be  summarized  as: 
review  the  data  that  informs  HIV  prevention,  describe  how  to  use  such  data,  and  build  in  a 
feedback  loop.   Discussion  followed  focused  on  the  proposed  Strategies. 

Strategy  1  -  CbCs  Performance  Indicators 

Israel  stated  that  while  the  language  may  be  clear  enough  for  Committee  members  but  that 

it  needs  to  be  comprehensible  to  the  actual  people  involved  in  the  evaluation  process. 

He  also  asserted  that  the  provider  community  will  ask: 

o    "  What  does  the  proposed  evaluation  process  do  for  me?' 

What  does  their  program  get  out  of  it,  and 
o    "  What  do  I  have  to  do?' 

What  are  the  paperwork  requirements  they  will  have  to  fulfill. 
Mike  and  Frank  suggested  adding  making  the  information  "meaningful'  for  stakeholders. 
Frank  suggested  changing  "  Utilize  CDC  performance  indicators 'to  inform..."  to,  "Utilize  CDC 
performance  indicators  to  educate  ..." 
Tracey  suggested, "  Utilize  CDC  performance  indicators  in  meaningful  ways... " 
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•  She  also  suggested  ensuring  that  the  performance  indicators  (Pis)  are  useful,  or  not,  to 
meeting  the  goal. 

Mike  suggested  that  the  Pis  be  molded  into  measurements  that  are  useful  to  SF's  efforts 
to  prevent  the  spread  of  HIV. 

•  Frank  said  the  Committee  needs  to  explain  how  to  make  good  use  of  the  Pis. 

■=>     He  added  that  it  needs  to  be  clear  that  Pis  shouldn't  be  seen  as,  and  needn't  be,  a 

burden  to  providers. 
Israel  explained  that  the  Pis  need  to  be  understood  as  important  to  providers: 
■=>    The  can  show  what  is  going  on  as  well  as  what  the  effects  are  of  the  prevention  efforts; 

and 
^>     They  could  provide  a  consolidated,  citywide  view  of  SF. 

He  highlighted  that  providers  focus  on  how  their  programs  are  doing  relative  to 
their  target  population(s);  but  the  Pis  can  give  an  overall  perspective. 
■=>     Frank  recommended  Israel's  suggestion  be  a  part  of  the  Introduction. 
Kym  stated  that, " Utilize  CDC  performance  indicators  to  inform  stakeholders"  or,  "to 
educate  and  inform,"  seems  one-sided. 
■=>     She  suggested  "Utilize  COC  performance  indicators  to  bring  recommendations  for 

discussion  with  stakeholders" 

•  Israel  said  that  the  Council  informs  the  CDC,  but  that  it  works  with,  interacts  with,  or 
dialogs  with  providers. 

•  Kym  suggested  "Utilize  CDC  performance  indicators  to  improve  HIV  prevention  efforts," 
taking  "stakeholders"  out  altogether. 

•  Lisa  pointed  out  that  the  performance  indicators  are  listed  in  Chapter  6:  Evaluation, 
Appendix  2  of  the  2004  SF  HIV  Prevention  Plan  (the  Plan). 

•  She  also  noted  the  Plan's  comments  regarding  the  purpose  of  evaluation  (Chapter  6: 
Evaluation,  Introduction,  Purpose  -  The  Plan,  Page  231): 

"This  chapter  also  attempts  to  reconcile  the  multiple  evaluation  requirements 
imposed  by  local,  state,  and  federal  institutions  with  the  need  to  keep  evaluation 
efforts  in  line  with  the  HPPC's  philosophy.   Evaluation  is  how  we  know  about  HIV  and 
HIV  prevention;  therefore,  it  is  essential  that  San  Francisco  find  a  way  to  meet 
these  requirements,  in  addition  to  collecting  locally  relevant  data,  while  avoiding 
unduly  burdening  service  providers  and  consumers.  One  of  the  first  tasks  the  HPPC 
will  take  on  in  2004  is  to  have  a  community  dialogue  about  these4  challenges,  using 
the  ideas  ..." 
It  was  noted  that  the  current  Status  reads, "  Unsure"  but  should  be, "  Working." 

Strategy  2  -  Community  Issues 

Frank  suggested  the  Strategy  read,  "Identify  emerging  issues,  conduct  surveys ..."  deleting 
the  term  "community" 

Kym  pointed  out  that  the  second  use  of  "community11  (in  "conduct  community  survey")  is 
necessary  because  it  limits  what  the  research  is  about. 

•  Israel  suggested  the  research  not  be  limited  to  SCANs. 

Tracey  suggested  adding  that  the  community  contributes  expertise. 

Kim  suggested  combining  the  text  of  Strategy  2  as  listed  in  the  two  drafts  distributed. 
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■=>     First  Draft:   "Conduct  Community  Surveys  that  allow  stakeholders  to  voice  expertise 
and  share  experiences" 
Second  Draft:   "Identify  emerging  issues,  conduct  community  surveys  (e.g.  SCAN), 
analyze  and  share  results" 

Strategy  3  -  Use  qualitative  findings 

Lisa  suggested  that  rather  than  the  Evaluation  Committee  " reviews  qualitative  findings.." 
the  action  read,  "Evaluation  Committee  makes  recommendations  on  how  to  analyze... " 

Strategy  4  -  Counseling  and  Testing  Data 

Frank  suggested  that  rather  than  it  reading, "  ...and  communicate  the  results  to  providers" 

it  read,  "share  (or  disseminate)  the  results.." 

Mike  said  that  counseling  and  testing  data  has  already  been  collected,  and  some 

communication  of  this  information  is  taking  place  with  providers,  however  there  is  little 

opportunity  for  stakeholders  to  share  their  expertise  (Objective  2.1)  or  for  a  strong 

feedback  loops  (Objective  2.2). 

o    He  highlighted  that  promoting  Objectives  2.1  and  2.2  is  what  needs  to  be  worked  on. 

o    He  suggested  ensuring  the  counseling  and  testing  data  is  'User-friendly,'  that  providers 
understand  what  the  data  is  and  how  to  use  it. 

He  noted  that  doing  so  is  requisite  to  feedback  loops. 

Israel  added  that  many  people  (providers  and  community  members)  don't  understand  that 

counseling  and  testing  data  is  reported  so  that  they  can  see  the  course  of  the  epidemic. 

Kym  suggested  adding  text  to  the  effect  of, "  creating  feedback  loops  so  that  useful 

information  is  shared" 

Mike  stated  that  the  issue  is  how  to  get  the  data  already  collected  out  to  the  community  so 

that  they  understand  and  use  it. 

■=>    He  added  that  part  of  this  could  be  the  format  of  the  data  communicated. 
•      Lisa  said  that  some  people  are,  understandably,  overwhelmed  by  the  quantity  of  data;  it 

should  be  reported  in  a  "digestible"  form. 

Kim  asked  about  combining  Strategies  1  (CDC's  performance  indicators)  4  (Counseling  and 

Testing  data)  as  "Making  available  data  useful'. 

o    Israel  pointed  out  that  counseling  and  testing  data  indicates  trends. 

Number  5  -  Monitoring  Reports 

Mike  asked  if  Monitoring  Reports  are  useful  to  providers. 

■=>     Frank  responded  that  he  reviews  them  and  makes  adjustments  based  on  what  he  sees. 

■=>    He  added  that  the  process  could  be  made  much  more  relevant  for  most  providers  by 

working  with/reviewing  the  reports  with  DPH  Program  Managers  who  have  an  excellent 

technical  understanding  of  the  data's  significance. 
Mike  suggested  reviewing  the  Monitoring  Report  process  for  next  year's  Council  agenda. 
Israel  stated  that  the  Monitoring  Reports  deal  with  data  too  far  in  the  past  to  be 
meaningful  and  relevant  to  most  providers. 

Israel  asked  what  is  learned  about  HIV  prevention  in  the  whole  of  SF  when/if  data  from  all 
the  Monitoring  Reports  is  put  together/consolidated. 
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o     He  highlighted  that  half  of  the  programs  in  SF  failed  the  cultural  competency 

component. 
^     He  questioned  why  data  is  being  collected  if  it  isn't  being  used  in  the  contract  process. 
Perry  asked  what  in  the  Monitoring  Report  is  useful. 

Mike  stated  that  SF  might  be  moving  to  standardized  contracts  across  all  departments. 
<=>     He  asked  if  standardized  monitoring  could  be  useful  in  ensuring  the  data  is  useful. 
Tracey  suggested  the  Committee  needs  to  find  ways  of  communicating  any  meaningful 
results  of  data  generated/collected  from  Monitoring  Reports. 

Frank  suggested  that  rather  than  "Monitoring  Reports"  this  Strategy  address  "Monitoring 
too/s"  which  includes  site  visits,  tools,  strategies,  etc. 

Israel  noted  that  the  contract  Monitoring  Report  covers  a  wide  range  of  topics  including: 
cultural  competence,  client  satisfaction,  etc. 

Mike  asked  if  client  satisfaction  should  be  a  standardized  (numeric)  scale. 
o    Israel  stated  that  perhaps  part  of  this  Goal  is  about  establishing  a  standard  (or 

standards). 

Strategy  6  -  Utilize  Required  Evaluation  Activities  to  Improve  Programs 

Kym  observed  this  Strategy  is  about  using  evaluation  data  to  make  programs  more 

effective,  and  that  such  requires  a  feedback  loop  to  providers. 

<^>     David  Weinman  asked  if  what  is  being  described  is  a  "Continuous  Quality  Improvement 
Model'  -  a  business  term  commonly  used  in  Quality  Assurance/Management. 
•      Tracey  said  the  strategy  should  also  ensure  evaluation  data  is  put  into  a  citywide  context. 

Israel  stated  that  the  ELI  system  is  probably  never  going  to  be  useful  in  meeting  these 

objectives. 


Goal  3  -  Innovations 


Kym  reviewed  the  draft  proposal. 

Lisa  noted  that  the  Evaluation  Committee  shouldn't  have  ongoing  responsibilities  in  the 

Action  Steps  (for  the  reasons  cited  above). 

<=>     Kym  noted  that  the  next  draft  would  show  the  HPPC  taking  action. 

Strategy  1  -  Use  Research  Data 

•      Israel  asked  about  the  term, " promising  practices." 

<=>     Kym  explained  that  those  are  practices  that  look  good  although  not  all  of  the  data  has 

been  collected,  analyzed,  and/or  reported. 
■=>    She  added  that  once  verified, "  promising  practices"  become, "  best  practices." 
Israel  suggested  changing  text  to  read,  "Use  data  on  interventions..."  rather  than  "Use 
research  on  interventions..."  so  that  the  Council  doesn't  have  to  wait  for  research,  to  be 
completed  and  reported  to  make  adjustments. 

Strategy  2  -  Conduct  Literature  Review 

Frank  suggested  providing  more  explanation  of  what  is  intended  for  the  literature  reviews. 
In  response  to  Israel's  question,  Kym  suggested  literature  reviews  be  conducted  annually. 
Tracey  noted  that  the  Status  should  read,  "Happens  when  Plan  is  updated." 

Strategy  3-  Recommendation  of  New  Prevention  Efforts 
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Frank  and  Kym  suggested  that  Strategy  3  could  be  consolidated  with  Strategy  1. 

Strategy  4  -  Recommendation  to  Create  and  Support  New  Prevention  Systems 

Frank  and  Kym  suggested  that  Strategy  4  could  be  consolidated  with  Strategy  1. 

Goal  3  Overall 

•      Mike  suggested  reversing  the  order  of  Strategies  1  &  2. 

Tracey  suggested  adding  Community  Forums  to  the  recommended  actions. 

^    Lisa  noted  that  the  2001  Plan  called  for  Community  Forums  to  explore  and  report  on 
developments  in  the  epidemic. 

Tracey  and  Frank  suggested  including  something  about  the  link  between  prevention  and 

structural  well  being. 

Perry  stated  that  the  Committee  didn't  seem  settled  on  Goal  3  and  suggested  meeting 

separately  to  discuss  it  further. 

<=>    Frank  suggested  discussing  Goal  3  at  the  07/01/04  Committee  meeting. 

■=>    Tracey  suggested  members  ruminate  on  Goal  3  and  be  prepared  to  discuss  it  further  at 
the  next  meeting. 
General  agreement  was  reached  to  further  discuss  Goal  3  at  the  07/01/04  Committee  meeting. 

6.    Next  Steps,  Evaluation,  and  Closure 

Mike  suggested  reviewing  all  three  Goals  at  the  07/01/04  meeting. 
Members  agreed  to  review  and  discuss  all  three  Goals  at  the  next  meeting. 

Israel  asked  about  the  three-year  Plan  process. 

<^>    Tracey  explained  that  the  first  year  is  preparation,  the  second  implementation,  and  the 

third  review. 
■=>    Israel  said  that  the  Committee  needs  to  ensure  the  evaluation  process  informs  the  Plan. 

Kym  suggested  and  there  was  general  agreement  to  review  the  timeline  at  the  next  meeting. 

It  was  noted  that  evaluation  would  be  by  means  of  Zoomerang. 

The  meeting  adjourned  5:01  PM. 


The  next  meeting  is  scheduled  for  Thursday,  July  1,  2004 
FROM  3:00  TO  5:00  PM  -  25  Van  NESS  AVENUE,  SUITE 330B. 


Minutes  were  prepared  by  David  Weinman,  Reviewed  by  Tracey  Packer  and  Perry  Rhodes  III. 
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3:00  pm  -  5:00  pm 

25  Van  Ness  Avenue,  330B 

^  San  Francisco 


AGENDA 

1.  Welcome  and  Announcements  3:00-3:10 

2.  Public  Comment  3:10-3:20 

3.  Approval  of  Minutes  from  June  3,  2004  3-20-3:25 

4.  Report  from  Steering  Committee  3:25-3:35 

5.  Continue  Development  of  Strategic  Plan  (Possible  Vote)  3:35-4:00 
>   Goal  3 

6.  Develop  Timeline  for  Strategic  Plan  (Possible  vote)  4:00-4:50 

7.  Next  Steps,  Evaluation,  and  Closure  4:50-5:00 

Next  meeting  will  be  August  5,  2004  at  3=00  -  5:00  PM 


NOTE;  All  meetings  are  open  to  the  public  and  are  held  in  handicapped  accessible  facilities. 
Meeting  dates  and  times  are  subject  to  change,  please  verify  by  calling  Betty  Chan  Lew  at  554-9492. 

Know  your  rights  under  the  Sunshine  Ordinance;  Government's  duty  Is  to  serve  the  public,  reaching  its  decisions  in  full  view  of  the 
public.  Commissions,  boards,  councils,  and  other  agencies  of  the  city  and  county  exist  to  conduct  the  people's  business.  This 
ordinance  assures  that  deliberations  are  conducted  before  the  people  and  that  City  operations  are  open  to  the  people's  review.  For 
more  Information  on  your  rights  under  the  sunshine  ordinance  or  to  report  a  violation  of  the  ordinance,  contact  the  Sunshine 
Ordinance  Task  Force.  Donna  Hall,  City  Hall,  Room  244,  1  Dr.  Carlton  B.  Goodleti  Place,  San  Francisco,  CA  94102,  Phone:  554- 
7724,  Fax:  554-7854,  E-Mail:  Donna^Hall&ci.sf.ca.us. 

In  order  to  assist  with  the  City's  efforts  to  accommodate  persons  with  severe  allergies,  environmental  illnesses,  multiple  chemical 
sensitivity  or  related  disabilities,  attendees  at  public  meetings  are  reminded  that  other  attendees  may  be  sensitive  to  various 
chemical  based  products.  Please  help  the  City  accommodate  these  individuals. 
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SAN  FRANCISCO 
Members  Present:  Lisa  Reyes  (HPS),  Israel  Nieves-Rivera,  and  Frank  Strona.  PUBLIC  LIBRARY 

Members  Absent:   Chata  Ashley,  Michael  Discepola  (on  leave),  Perry  Rhodes,  III,  and  Marcel 
Miranda. 

Professional  Staff:   Kym  Dorman  (Harder  &  Co),  Tracey  Packer  (HPS),  and  David  Weinman  (Note 
Taker). 

1.  Welcome  and  Announcements 

Perry  Rhodes,  III  asked  Tracey  Packer  to  facilitate  in  his  absence.   She  called  the  meeting  to 
order  at  3:10  PAA. 

2.  Public  Comment 

There  was  no  public  comment. 

3.  Approval  of  Minutes  from  June  3,  2004 

Motion  was  made  and  seconded  to  accept  the  minutes  from  the  meeting  06/03/04.  The 
members  present  expressed  approval  and  Kym  Dorman  said  that  they  were  helpful  to  her.  No 
objections,  corrections,  or  changes  were  offered.  The  minutes  were  approved. 

4.  Report  from  Steering  Committee 

Tracey  and  Israel  Nieves-Rivera  provided  an  overview  — 

•  The  07/08/04  Council  meeting  will  include  presentations  by  representatives  of  CDC  directly 
funded  agencies. 

•  HPS  hasn't  received  the  CDC's  guidance  for  the  Cooperative  Agreement  Application. 

o     Once  received,  the  Council,  with  the  help  of  the  DPH  will  review  the  priorities  in  the 

application,  and  then  draft  and  approve  a  letter  of  concurrence. 
■=>    The  application  is  due  sometime  in  September, 
o     The  CDC  has  informed  the  DPH  that  the  guidance  should  be  sent  in  the  near  future. 

-      Israel  offered  to  email  the  CDC  if  DPH  doesn't  get  it  by  Tuesday  (07/06/04). 
The  HIV  Planning  Leadership  Summit  (HPLS)  is  scheduled  for  SF  next  year. 
■=>     Organizers  are,  however,  having  difficulty  finding  a  hotel  large  enough  to  accommodate 

the  event  with  such  short  notice. 
There  was  discussion  regarding  whether  Zoomerang  works  as  process  evaluation. 
<=>     The  number  of  people  responding  has  been  very  low  and  may  not  be  sufficient. 
<=>    There  should  be  either  a  recommitment  from  members  or  reversion  to  paper  forms. 

5.  Continue  Development  of  Strategic  Plan 

Kym  distributed  the  document  entitled,  "Changing  the  Culture  of  Evaluation  in  SF  HIV 
Prevention  2005-2007 STRA  TEGIC  PLAN  FOR  HIV  PREVENTION  EVALUA  TION  IN  SF  (Draft 
June  29,  2004)"  a  copy  of  which  is  available  to  absent  members  upon  request.  The  following 
provides  highlights  of  the  discussion  by  topic/section  of  the  Strategic  Plan. 


Flow  Chart  (page  3),  including  the  thumbnail  descriptions  of  the  three  goals. 
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Several  members  expressed  concern  with  the  wording  of  Goal  1:  "  Creating  Feedback  Loops" 

<=>     A  number  of  suggestions  were  made,  including  incorporating  " '  bridges'  'in  the  thumbnail. 

<=>     Kym  will  work  on  rewording  this  thumbnail  description. 
Frank  summarized  the  three  steps  as:  Gathering  the  Story,  Sharing  or  Bridging  the  Story,  and 

I  earning  from  the  Story.  _ 

Review  of  Goal  2  -  "Creating  Feedback  Loops" 

Kym  highlighted  the  changes  made  in  Goal  2,  particularly  the  addition  of  " Coordinator" and 

"Contributor"  in  the  column  showing  who  is  responsible. 
As  these  changes  were  results  of  discussion  during  previous  meetings,  it  was  agreed  to  review 

the  text  of  Goal  2  later,  if  time  permits. 

Goal  3  -  Innovation  for  Change 

Kym  pointed  out  that  throughout  the  Strategic  Plan  the  wording  has  been  changed  from  the 

committee, " doing"  to  the  committee  "recommends"  processes  and/or  procedures. 

Strategy  3.1  -  Literature  Review 

Israel  asked  why  Goal  3  includes  conducting  a  literature  review,  suggesting  such  a  review 

should  be  in  Goal  2  (Creating  Feedback  Loops). 

■=>     He  also  noted  that  Goal  1  (Telling  the  Story)  also  includes  conducting  research  on 
interventions  (Strategy  1.1)  and  asked  if  this  isn't  the  literature  review. 

Frank  suggested  that  the  literature  review  in  part  of  an  ongoing  evaluation. 

Tracey  suggested  Strategy  3.1  be  about  disseminating  and  documenting  the  changes  and/or 

innovations. 

Kym  said  that  literature  review  isn't  clearly  delineated  anywhere. 

<=>    She  added  that  the  review  here  is  to  be  sure  that  innovative  approaches  are  effective. 

Frank  suggested  that  this  literature  review  leads  into  Strategy  1.1  (Conduct  research  on 

interventions). 

Kym  said  that  this  is  a  gathering  of  information  that  enables  telling  the  story  (Goal  1). 

Israel  proposed  rather  than  a  literature  review,  Strategy  3.1  be  a,  "Summary  of  Findings.  " 

It  was  agreed  that  Strategy  3.1  is  a  summary  of  what  has  been  found. 

Strategy  3.2a  -  Promising  Practices  and  Innovative  Approaches 

Frank  suggested  that  there  are  two  types  of  new  interventions: 
Those  addressing  gaps  in  current  efforts;  and 
Innovative  /  new  work. 

Israel  noted  that  sometimes  the  gap  isn't  that  a  group  isn't  being  addressed,  but  that  a 

particular  intervention  hasn't  been  applied  to  that  population. 

Frank  said  that  sometimes  the  gap  is  the  recognition  that  something  doesn't  work. 

<=>     He  added  that  if  something  doesn't  work  it  should  be  "defunded. " 

Israel  said  that  there  needs  to  be  evaluation  whether  what  is  being  done  works. 

■=>    If  it  doesn't  work,  he  added,  the  process  should  ask,  "What  about  it  isn't  working?" 

o     He  also  suggested  comparing  interventions  -  where  and  how  they  are,  and  aren't  working. 

Frank  said  that  determining  what  is  working  could  be  dependent  on  what  part  of  the  data  is 

being  looked  at/evaluated  and  should  focus  on  the  results. 
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He  cited  the  example  of  a  web-site  getting  5,000  hits,  but  only  netting  three  new 
members;  it  is  the  new  members  that  are  important,  not  the  hits. 
<=>    Israel  added  that  there  should  be  a  model  of  trickle  down  of  data  examined,  citing 
Rafael  Diaz's  project  that  interviewed  thousands  of  people  to  qualify  an  "n"  of  300. 

•  Frank  said  that  a  practice  might  be  good  with  one  group,  but  ineffective  elsewhere. 
Israel  and  Frank  both  suggested  that  Strategy  3.2  is  about  both:  shortcomings  of  data 
collection,  as  well  as  best  practices. 

•  Tracey  said  that  Strategy  3.2  is  also  about  the  challenges  of  prevention. 

Israel  said  that  information  used  to  determine  which  tactics  are  working,  or  not  working, 
needs  to  come  from  multiple  sources  such  as  ELI,  Community  Forums,  and  the  like. 
Frank  said  that  the  HPPC  needs  to  take  a  stand  and  stop  being  wishy-washy  about  effective 
and  ineffective  interventions. 

Wording  of  Strategy  3.2 

•  Frank  suggested  Strategy  3.2  could  be  worded  as, " Best  practices,  challenges,  and 
ineffective  practices"  which  would  be  better  than  using  a  term  like, "  Bad practices" 

■=>     The  strategy,  he  added,  should  be  ensuring  that  agencies  aren't  discouraged  from  trying 
new  things,  even  if  they  ultimately  prove  to  be  ineffective. 

•  Kym  suggested  changing  the  wording  to  something  like, u  Challenges  and  overcoming 
challenges" 

Amendments  to  Goal  3  -  Strategies  and  Action  Steps 

It  was  suggested  that  a  new  Strategy  1.1  be  added  -  n  Conduct  literature  review  on 
interventions  and  evaluations" 

o    The  older  Strategies  would  then  change  number:  old  1.1  becomes  1.2,  etc. 
Israel  suggested  adding  a  Strategy  (3.4)  to  identify  gaps  in  prevention  efforts. 
■=>    This  new  Strategy  would,  he  added,  lead  back  to  Strategies  1.1  and  1.2  (Conduct 

Research  on  Interventions), 
o     Kym  added  that  this  new  strategy  could  encompass  recommendations  for  change, 
including  identifying  issues  that  need  other,  better,  practices  applied. 

She  also  suggested  creating  documentation  that  gives  information  on  how  agencies 
have  overcome  gaps  and  other  challenges. 

•  Israel  pointed  out  that  some  agencies  are  still  doing  the  same  interventions  they  have  been 
doing  for  years  without  regard  to  the  bottom  line  impacT  it  is  having  on  transmission  rates. 

Agreements 

With  the  understanding  that  Kym  will  do  word-smithing,  the  Committee  agreed  that: 

■=o    Objective  3.2  becomes, "  Identify  gaps,  challenges,  how  to  overcome  challenges,  needs 

and  ineffective  practices" 
^    Objective  3.3  becomes, "  Take  promising  practices  to  recommend  new  prevention 

approaches." 
^>    Objective  3.4  becomes  old  Objective  3.3, n  Utilize  findings  to  recommend  new 

prevention  efforts" 
o    Strategy  3.2  A)  becomes  3.3  A)  and  should  read  ""recommend'  rather  than  "use" 
■=>    Strategy  3.3  A)  become  3.4, 
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Develop  Timeline  for  Implementation  of  Strategic  Plan 

Kym  and  Tracey  explained  the  timeline  as  the  schedule  the  Committee  will  use  to  complete  its 
work  for  the  year.   They  highlighted  the  steps  as: 

The  Committee  decides  on  the  Strategic  Plan's  processes; 
The  Committee  presents  its  proposal  to  the  Council; 
The  Council  reviews,  amends  and  approves  the  Strategic  Plan;  and 
The  Strategic  Plan  is  implemented. 
Israel  and  Kym  asked  what  happens  if  the  Committee  doesn't  have  time  to  make 
recommendations  in  all  of  the  places  where  the  Strategic  Plan  calls  for  it  to. 
■=>    Tracey  pointed  out  that  not  all  recommendations  need  to  come  from  the  committee, 

that  other  processes  could,  and  should  be  used. 
Tracey  pointed  out  that  the  Strategic  Plan  also  relates  to  the  RFP. 
<=>    To  meet  the  RFP's  timeframe  the  Committee  needs  to  present  the  plan  to  the  Council 

and  make  any  changes  required  in  October. 
«=>    She  noted  that  the  Strategic  Plan  also  relates  to  the  implementation  of  PEMS. 
Frank  suggested  the  August  meeting  be  a  brainstorming  session  regarding  the  connection 
between  Goals  and  RFP  followed  by  a  long  meeting  (in  September?)  to  finalize  everything. 
o     The  longer  meeting  would  allow  a  final  review  of  the  plan  in  one  sitting,  as  the 

Committee  tends  to  go  back  to  previous  discussions  at  the  beginning  of  each  meeting. 
Tracey  suggested  Kym  send  the  changes  from  this  meeting  to  all  members  and  that  they 
commit  to  sending  back  their  comments  with  a  proposed  timeline. 
Frank  suggested  creating  a  specific  schedule  of  events. 

Tracey  suggested  the  Committee  look  at  the  objectives  relative  to  the  RFP  and  use  the 
August  meeting  to  review  the  technical  assistance  (TA)  needs  of  providers  for  each  area. 
o     Frank  pointed  out  that  agencies'  staff  change  with  some  frequency,  so  it  would  be 

difficult  to  identify  what  is  needed. 
■=>    Lisa  pointed  out  that  HPS  Program  Managers  should  know  what  the  needs  are. 
Frank  and  Tracey  suggested  going  through  each  Action  Step  to  detail  how  to  proceed. 

Agreement 

It  was  agreed  that  the  Committee  have  a  second  meeting  in  July.  The  proposed  dates  are: 

o     Tuesday  07/27/04  -  3:00  to  5:00  PAA,  or  alternatively 

o    Thursday  07/29/04  -  3:00  to  5:00  PM 

o    [update:    the  next  meeting  will  be  July  27,  3-5  PM] 
The  next  regular  meeting  is  scheduled  for  08/05/04. 
Tracey  and  Lisa  will  send  email  and  phone  members  to  determine  which  date  is  best  and  to 
encourage  attendance. 

Frank  suggested  pointing  out  that  if  the  Committee  gets  through  all  of  the  Strategic  Plan  it 

could  cance\  the  September  meeting  as  a  "carrot." 
Kym  will  have  the  Goals  updated  by  07/13/04,  two  weeks  in  advance  of  the  next  meeting. 
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7.    Next  Steps,  Evaluation,  and  Closure 

It  was  noted  that  evaluation  would  be  by  means  of  Zoomerang. 


The  meeting  adjourned  4:45  PAA. 


The  next  meeting  is  scheduled  for  Tuesday,  July  27,  2004 

FROM  3:00  TO  5:00  PM  -  25  Van  NESS  AVENUE,  SUITE  330B. 


Minutes  were  prepared  by  David  Weinman,  Reviewed  by  Tracey  Packer  and  Lisa  Reyes. 
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AGENDA 


1.    Welcome  and  Announcements 


2.  Public  Comment 


3.  Approval  of  Minutes  from  July  1st,  2004 

4.  Report  from  Steering  Committee 

5.  Continue  Development  of  Strategic  Plan  (Possible  Vote) 

6.  Develop/Review  Timeline  for  Strategic  Plan  (Possible  vote) 

7.  Next  Steps,  Evaluation,  and  Closure 


3:00-3:10 
3:10-3:20 
3:20-3:25 
3:25-3:35 
3:35-4:00 
4:00-4:50 
4:50-5:00 


Next  meeting  will  be  August  5,  2004  at  3:00  -  5:00  PM 


NOTE:  All  meetings  are  open  to  the  public  and  are  held  in  handicapped  accessible  facilities 
Meeting  dates  and  times  are  subject  to  change,  please  verify  by  calling  Betty  Chan  Lew  at  554-9492, 

Know  your  rights  under  the  Sunshine  Ordinance:  Government's  duty  is  to  serve  the  public,  reaching  its  decisions  in  full  view  of  the 
public.   Commissions,  boards,  councils,  and  other  agencies  of  the  city  and  county  exist  to  conduct  the  people's  business.   This 
ordinance  assures  that  deliberations  are  conducted  before  the  people  and  that  City  operations  are  open  to  the  people's  review.  For 
more  information  on  your  rights  under  the  sunshine  ordinance  or  to  report  a  violation  of  the  ordinance,  contact  the  Sunshine 
Ordinance  Task  Force.  Donna  Hall,  City  Hall,  Room  244,  1  Dr.  Carlton  B  Goodlett  Place,  San  Francisco,  CA  94102.  Phone:  554- 
7724,  Fax    554-7854.  E-Mail:  Donna_Hall@ci.sfca. us 

In  order  to  assist  with  the  City's  efforts  to  accommodate  persons  with  severe  allergies,  environmental  illnesses,  multiple  chemical 
sensitivity  or  related  disabilities,  attendees  at  public  meetings  are  reminded  that  other  attendees  may  be  sensitive  to  various 
chemical  based  products.   Please  help  the  City  accommodate  these  individuals. 
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Thursday,  August  5,  2004 

3:00  pm  -  5:00  pm 
25  Van  Ness  Avenue,  330B 
^San  Francisco 

^AGENDA 

1.  Welcome  and  Announcements  3:00-3:10 

2.  Public  Comment  3:10-3:20 

3.  Develop  Proposal  for  Referral  Tracking  System  (possible  vote)  3:20-4:30 

4.  Review  Timeline  for  Strategic  Plan  (Possible  vote)  4:30-4:50 

5.  Next  Steps,  Evaluation,  and  Closure  4:50-5:00 


Next  meeting  will  be  September  2nd,  2004  at  3:00  -  5:00  PfA 


NOTE:  All  meetings  are  open  to  the  public  and  are-  held  In  handicapped  accessible  facilities. 
Meeting  dates  and  times  are  Subject  to  change,  please  verify  by  celling  Betty  Chan  Lew  at  554-9492. 

Know  your  rights  under  the  Sunshine  Ordinance.  Government's  duty  is  to  serve  the  public,  reaching  its  decisions  in  full  view  of  the 
public.  Commissions,  boards,  councils,  and  other  agencies  of  the  city  and  county  exist  to  conduct  the  people's  business.   This 
ordinance  assures  that  deliberations  are  conducted  before  the  people  and  that  City  operations  are  open  /o  the  people's  review.   For 
more  information  on  your  rights  under  the  sunshine  ordinance  or  to  report  a  violation  of  the  ordinance,  contact  the  Sunshine 
Ordinance  Task  Force    Donna  Hall,  City  Hall.  Room  244,  1  Dr.  Carlton  6  Goodlett  Place,  San  Francisco,  CA  94102,  Phone:  554- 
7724,  Fax:  554-7854,  E-Mail.   Donna_Hall@ci.Sf.C3. us. 

In  order  to  assist  with  the  Crty's  efforts  to  accommodate  persons  with  severe  allergies,  environmental  illnesses,  multiple  chemical 
sensitivity  or  related  disabilities,  attendees  at  public  meetings  are  reminded  that  other  attendees  may  be  sensitive  to  vanous 
chemical  based  products.  Please  help  the  City  accommodate  those  individuals. 
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Members  Present:  Marcel  Miranda,  Israel  Nieves-Rivera,  Perry  Rhodes,  III,  and  Frank  Strona. 

Members  Absent:  Chata  Ashley,  Michael  Discepola. 

Professional  Staff:   Tracey  Packer  (HPS),  Mike  Pendo  (HPS),  David  Weinman  (Note  Taker),  and 
Allison  Weston  (Harder  &  Co). 

1.     Welcome  and  Announcements 

Perry  Rhodes,  III  called  the  meeting  to  order  at  3:16  PM.  He  invited  members  to  make 
announcements. 

•  Israel  Nieves-Rivera  explained  that  he  is  in  a  workgroup  putting  together  the  national 
evaluation  guidance  part  of  the  CDC's  "Advancing  HIV  Prevention  Initiative" 

o     He  would  be  happy  to  pass  along  members'  comments. 

•  Allison  reminded  members  that  she  is  sitting  in  for  Kym  Dorman  who  is  on  vacation. 


2.    Public  Comment 

There  was  no  public  comment. 


3.    Approval  of  Minutes  from  July  27,  2004 

Motion  was  made  and  seconded  to  accept  the  minutes  from  the  meeting  07/27/04. 

•  Tracey  pointed  out  that  because  of  the  short  turnaround  time  since  the  last  meeting  the 
draft  submitted  may  have  typos  and  notes  attached  that  would  be  expunged. 

•  David  Weinman  noted  that  the  date  listed  for  the  next  meeting  reads, "  Thursday  August 
6th"  but  should  read, "  Thursday  August  5th !' 

There  was  no  other  discussion.  No  other  corrections  or  objections  were  offered.  The  minutes 
were  approved. 


4.    Report  from  Steering  Committee 

Tracey  reviewed  the  draft  agenda  for  the  Council  meeting  next  week  08/12/04.  It  was  noted 
that  the  agenda  was  very  ambitious  for  a  single  meeting. 


Develop  Proposal  for  Referral  Tracking  System 

Perry  introduced  the  topic  by  reminding  members  that  the  professional  staff  distributed  by 
email  a  copy  of  Chapter  8  -  Linkages  from  the  1997  Plan  (CHAPT8.DOC). 

<=>     Tracey  reviewed  the  chapter. 
Perry  asked  members  for  examples  of  their  specific  experience  with  Referral  Tracking. 
Discussion  followed  as  highlighted  by  the  following  -  presented  by  topic. 


What  is  a  Successful  Referral? 
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Mike  suggested  defining  what  constitutes  a  successful  referral. 

Israel  noted  that  the  definition  is  being  grappled  with  nationally. 

<=>    He  noted  the  CDC  defines  a  successful  referral  as  being:  referrals  in  which  clients' 

contact  with  a  service  actually  occurs  and  is  verified, 
o    He  distributed  the  document  entitled, " Strategic  Referrals  for  HIV  Prevention:  Should 

SF  identify  which  interventions  are  appropriate  to  conduct  the  tracking  of  referrals?"  a 

copy  of  which  is  available  to  absent  members  upon  request. 
Israel  proposed  SF  practice  strategic  referral  tracking,  limiting  the  tracking  to: 
Those  interventions  which  can  reasonably  be  monitored, 
Those  interventions  worth  tracking  (as  they  are  not  all  equal),  and 
Those  interventions  which  would  benefit  from  being  tracked. 
Allison  asked  if  what  is  being  measured  relates  to  the  desired  outcome(s). 
<=>    Tracking  individual  referrals  may  not  provide  useful  information  as  regards  outcome, 

while  the  aggregate  of  attempted/accomplished  referrals  might. 
<=>    Useful  data  about  tracking,  she  added,  might  be  a  result  of  several  different  indicators, 

including  number  of  completed  referrals  -  but  not  limited  to  that  statistic. 


Barriers  to  Referral  Tracking 


The  following  areas  were  discussed  as  barriers  — 
Agencies'  erroneous  identification  as  bilingual. 
■=>    Although  some  staff  may  be  bilingual  they  may  not  be  available  on  the  front-line,  for  all 

hours  of  service,  or  involved  in  all  programs. 
Frank  Strona  noted  that  the  challenges  haven't  changed  from  the  1997  list  in  Chapter  8. 
Referral  at  Rapid  Testing  is  a  particularly  difficult. 
•      Confidentiality  provisions  often  prohibit  providers  from  reporting  on,  or  verifying,  that 
referrals  have  taken  place. 

■=>    Tracking  referrals  to  confidential  testing  is  nearly  impossible. 
<=>    Mike  suggested  doing  an  aggregate  of  referrals  to  testing. 

He  noted  that  Barbara  Adler  has  told  him  that  such  information  could  be  provided. 
■=>    It  was  noted  that  some  confidential  testing  sites  ask  and  record  where  clients  were 

referred  from,  and  others  don't. 
There  was  discussion  about  the  difficulties  of  assigning  "Unique  Identifiers' 'to  clients, 
o    Creating  and  maintaining  a  database  system  able  to  track  clients  requires  such. 
■=>    If  the  Unique  Identifier  is  random,  it  becomes  unlikely  that  clients  will  remember  it. 
<=>    It  was  noted  that  the  CDC  supports  including  a  Unique  Identifier  that  comes  from 

clients'  information  and  which  enables  providers  to  recreate  the  code  when  clients' 

present  themselves  for  service. 
o    The  Unique  Identifier  may  not  be  confidential  if  it  is  based  on  the  clients'  data  (i.e.,  all 

or  part  of  the  Social  Security  Number). 
Israel  said  that  it  is  important  to  track  referrals  to  non-prevention  providers;  i.e.,  care 
services,  substance  abuse,  and  mental  health. 
^>     He  noted  the  difficulty  of  such  tracking,  including  that  non-prevention  providers  have 

their  own  data  collection  systems,  which  may,  or  may  not,  interface  with  ELI  or  PBMS. 
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<=>     He  cited  LA's  experience  where  Care  and  Prevention  are  on  the  same  system  [the  HIV 

Information  Referral  System  (HIRS)]  and  they  exchange  encrypted  identifiers. 
There  was  discussion  of  the  confidentiality  issues  surrounding  in  tracking  referrals  relating 
to  Prevention  Case  Management  (PCM). 

Barriers  in  Tracking  Outreach 

Alison  noted  that  she  is  consulting  with  an  Outreach  Working  Group  whose  charge  includes 

looking  at  Referral  Tracking. 

<=>     She  suggested  asking  participating  providers  what  they  are  already  doing,  what  is 

working,  and  for  their  views  on  having  to  track  referrals. 
■=>    It  was  also  noted  that  the  Working  Group  would  propose  a  definition  of  outreach. 
Marcel  questioned  if  tracking  street  outreach  could  be  practical,  noting  that  it  is  very 
difficult. 

Mike  and  Perry  observed  that  outreach  workers  often  have  creative  ways  to  track 
referrals,  including  using  Cell  Phones  and  accompanying  clients  to  a  service  providers. 
>=>    Many  outreach  workers  have  the  attitude  that  they  will  see  the  people  again. 


Resource  Guides 


Israel  suggested  that  resource  guides  include: 

When  and  where  bilingual  staff  is  available, 

Which  programs  have  bilingual  support,  and 

Contact  information  for  specific  languages. 
Frank  suggested  resource  guides  should  include  if  the  agency  is  willing  to  serve  other 
agencies'  populations;  and  if  the  agency  is  open  to  collaborations. 

Several  members  commented  on  the  advantages  of  an  online  resource  guide,  saying  it  would 
be  useful,  accessible,  and  easy  to  keep  updated. 

RFP 

Frank  suggested  using  the  RFP  to  specify  Referral  Tracking  and  its  evaluation/monitoring. 

Marcel  said  that  the  new  RFP  should  promote  collaboration  by  encouraging  one-to-one 

relations  between  staff  at  different  agencies. 

Mike  suggested  encouraging  innovative  interventions  in  the  RFP  by  having  providers  devise 

their  own  evaluation  techniques. 

c>     Frank  added  that  by  doing  this,  the  DPH's  contract  managers  would  have  to  develop  a 

new  role  helping  agencies  develop  interventions  as  well  as  means  of  monitoring. 
Marcel  Miranda  proposed  making  Referral  Tracking  a  priority. 


Mechanisms/Process  of  Referral  Tracking 


Frank  asked  if  individual  referrals  have  to  be  tracked,  or  if  the  tracking  could  be  of  the 

total  number  of  referrals. 

Marcel  suggested  a  online  referral  system  that  could  include: 

Searchable  by  population  served,  location,  and  service(s)  provided; 

Printing  out  where  the  referral  is  (address,  phone,  map)  for  the  client;  and 

Tracking  of  the  referral  by  the  system. 


HIV  PREVENTION  PLANNING  COUNCIL  (HPPC) 

Evaluation  Committee 

Action  Minutes  From  Meeting: 

August  5,  2004 


Marcel  noted  that  to  simplify  tracking  Glide  keeps  separate  lists  of  referrals  to  different 

services. 

He  also  said  that  Glide  is  working  on  a  way  to  provide  incentives  for  completed  referrals. 

Mike  suggested  monitoring  different  referral  methods  to  evaluate  which,  if  any,  work. 

Frank  and  Tracey  noted  that  while  the  ELI  system  has  some  facility  to  collect  data  about 

referrals,  PEMS  may  be  more  comprehensive,  flexible,  and  customizable. 

Israel  suggested  the  Council  agree  upon  a  small  number  of  referrals  to  track  citywide. 

Performance  Indicators 

There  was  discussion  regarding  Performance  Indicators  (PI),  touching  on  these  issues: 
■=>    The  need  to  avoid  contradicting  the  CDC's  current  and  future  requirements. 

There  was  agreement,  however,  that  SF  HPPC  should  not  wait  for  the  CDC's 

direction  but  should  proceed  with  meeting  local  needs. 
■=>    The  CDC's  current  PI  are  established,  and  included  in  the  2004  Plan. 

The  Council's  role  is  to  ensure  good  faith  efforts  are  made  to  comply  with  the 

funders'  requirements. 
^    Providers  could  develop  their  own  indicators,  in  addition  to  the  established  set. 
■=>    The  DPH  should  provide  technical  assistance  to  agencies  helping  them  develop  their  own 

indicators  as  well  as  to  work  with  those  established  by  funders. 
There  was  discussion  regarding  who  should  draw  up  requirements  beyond  those  of  the  CDC, 
including  these  points: 
<=>    The  Council  could  create  a  list  of  requirements. 

Objection  was  raised  as  the  Council's  role  is  to  prioritize  funding,  not  to  create 

requirements. 
<=>    The  Council  could  direct  the  AIDS  Office  to  establish  requirements. 
<=>    Providers  could  be  given  a  set  of  PI  to  choose  from. 
<=>    Providers  could  create  their  own  requirements. 

Individually,  or  with  the  help  of  DPH  Contract  Managers. 
■=>    Providers  could  combine  their  own  customized  PI  with  those  offered  to  choose  from. 
It  was  suggested  the  Committee  create  a  matrix  evaluating  Referral  Tracking  that: 
<=>    Monitors  the  PI  specified  by  funders, 
■=>    Oversees  the  PI  devised  by  providers,  and 

<=>    Includes  a  review  of  both  types  of  indicators  by  DPH's  Contract  Managers. 
Several  members  suggested  the  HPPC  should  keep  ahead  of  the  CDC's  requirements. 


Where  to  start? 


Mike  suggested  absent  research  data,  the  Committee  should  decide  what  it  thinks  will  work 

and  track  that. 

Marcel  suggested  starting  Referral  Tracking  in  a  simple,  doable  way  as  a  means  of  building 

up  experience  and  knowledge  of  what  to  do  and  how  to  do  it. 

■=>    He  demonstrated  by  showing  Glide's  new  referral  calling  cards: 

Cards  are  distributed  to  providers  participating  in  the  tracking  exercise; 

Cards  are  marked  as  coming  from  the  referring  agency  (internal  or  external); 

The  providers  give  clients  a  card  when  referring  them  to  Glide's  services;  and 
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Periodically,  the  cards  are  counted  and  tallies  generated. 

o     He  added  that  the  Committee/Council  should  let  providers  know  what  was  considered. 

Israel  suggested  starting  with  one  of  the  areas  highlighted  in  the  document  he  distributed: 

Outreach,  Counseling,  Outreach  Events,  Groups,  City  Wide,  and  Counseling  and  Testing. 
•      Frank  suggested  that  agencies  funded  for  several  interventions  should  start  by  tracking 

only  the  most  effective. 
Agreement  was  reached  that  providers  should  choose  which  intervention(s)  to  track  -  that 
tracking  all  referrals  is  not  practical. 


Proposed  Referral  Tracking  Process: 


The  members  jointly  drew  up  the  following  four-step  process: 

1.  Choose  an  Intervention 

2.  Choose  a  method  for  tracking  referrals 

3.  Evaluate  the  process  for  the  first  year 

4.  Resources 

Discussion  and  clarification  followed. 

1.  Choose  an  Intervention 

o    It  was  agreed  that  the  referrals  to  be  tracked  should  be  funded  interventions. 
&    Perry  said  that  providers  should  have  confidence  that  the  DPH  won't  override  their 

choice. 
■=>    The  process  should  ensure  that  providers  don't  all  track  the  same  type  of  referral. 
o    Tracey  suggested  including  narrative  in  the  RFP  explaining  that  some  referrals  would  be 

tracked  and  alerting  providers  to  be  prepared. 
<=>     Frank  said  that  different  interventions  could  be  tracked  during  the  next  monitoring 

period. 
o    Israel  suggested  narrative  making  it  clear  that  Referral  Tracking  is  new  and  that  the 

monitoring  will  be  used  to  develop  effective,  meaningful  tracking. 

2.  Choose  a  Method  for  Referral  Tracking 

■=>    A  number  of  methodologies  of  tracking  were  discussed  (including:  referral  cards,  and  an 
Internet-based  tracking  system)  and  the  members  agreed  to  offer  these  processes: 
A  Menu  of  methods  providers  could  choose  from;  and/or 
Providers,  working  with  HPS,  develop  their  own  methods. 
■=>    It  was  suggested  that  focus  groups  could  be  held  with  providers  to  inform  the  menu 
choices. 

3.  Evaluate  the  Process  for  the  First  Year. 

■=>     Frank  suggested  including  in  the  RFP  provisions  for  annual  reviews  of  the  effectiveness 

of  providers'  Referral  Tracking, 
o    It  was  suggested  that  programs  are  to  be  evaluated  based  on  developing  and 

implementing  a  tracking  system,  rather  than  the  results  of  the  tracking. 
Particularly  as  there  is  no  comparison  data  available. 
■=>    It  was  noted  that  while  evaluation  is  important,  it  is  often  seen  as  punitive,  and 

providers  need  to  be  encouraged. 
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Marcel  said  that  after  a  year  it  should  be  clear  which  types  of  Referral  Tracking  work. 
He  added  that  the  review  should  also  suggest  improvements  and  enhancements. 
^     Tracey  suggested  establishing  a  Committee  to  review  tracking  data  collected. 
&    She  highlighted,  however,  that  it  can  be  difficult  to  change  things  in  the  middle  of  a 

contract  period. 
<=>    Israel  said  that  the  RFP's  narrative  should  explain  that  the  process  would  include  annual 

reviews/evaluations  as  well  as  ongoing  modifications/improvements. 

4.  Resources 

■=>    There  was  discussion  regarding  the  funding  of  Referral  Tracking,  including  whether  to 

include  the  cost  by  shrinking  services,  or  add  it  as  a  cost  item. 
■=>    Mike  suggested  pairing  staff  from  the  Applied  Research  Unit  to  each  agency  for 
assistance  in  evaluating  referral  tracking. 

Motion  was  made  and  seconded  to  approve  four-step  process  described.  There  was  no  further 
discussion.  There  were  no  objections  or  abstentions.  The  motion  was  agreed  by  consensus. 


Review  Timeline  for  Strategic  Plan 

It  was  agreed  that  during  the  09/02/04  meeting  the  Committee  would  discuss  Strategies: 

1.2  B)     Provide  technical  assistance  to  providers  to  fulfill  evaluation  requirements  ... 

1.3  A)     Make  evaluation  useful  and  meaningful. 

1.4  A)     Development  of  standardized  outcomes/performance  measures. 
2.1  A)     Use  of  CDCs  performance  indicators. 


7.    Next  Steps,  Evaluation,  and  Closure 

The  meeting  adjourned  4:56  PM. 


THE  NEXT  MEETING  IS  SCHEDULED  FOR  THURSDAY,  SEPTEMBER  2nd,  2004 
FROM  3:00  TO  5:00  PM  -  25  Van  NESS  AVENUE,  SUITE 330B. 


Minutes  were  prepared  by  David  Weinman,  Reviewed  by  Tracey  Packer  and  Perry  Rhodes  III. 
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Members  Present:  Michael  Discepola,  Israel  Nieves-Rivera,  Perry  Rhodes,  III,  and  Lisa  Reyes. 

Members  Absent:  Chata  Ashley,  Marcel  Miranda,  and  Frank  Strona. 

Professional  Staff:  Kym  Dorman  (Harder  &  Co),  Tracey  Packer  (HPS),  Mike  Pendo'(HPS),  David 
Weinman  (Note  Taker),  and  Allison  Weston  (Harder  &  Co). 

1.     Welcome  and  Announcements 

Perry  Rhodes,  III  called  the  meeting  to  order  at  3:20  PM.  He  invited  members  to  make 
announcements. 

Perry  announced  a  fund  raising  event  10/16/04  for  Cal  PEP. 

Israel  Nieves-Rivera  pointed  out  that  this  Committee's  last  meeting  had  only  one  evaluation 

submitted  and  he  encouraged  members  to  respond  to  the  Zoomerang. 


2.    Public  Comment 

There  was  no  public  comment. 


3.    Approval  of  Minutes  from  August  5,  2004 

Minutes  were  approved  without  comment,  objection,  or  abstention. 


Discussion  of  Presentation  to  Council  09/09/04  (Added  to  Agenda) 

Perry  explained  that  in  light  of  the  Committee's  presentation  to  Council  next  week  (09/09/04) 
this  item  has  been  added  to  the  agenda.   Allison  Weston  distributed  a  draft  presentation 
entitled,  "Changing  the  Culture  of  Evaluation  in  5F  HIV  Prevention"  a  copy  of  which  is  available 
to  absent  members  upon  request.   She  explained  that  she  put  the  draft  together  as  a  starting 
point  for  the  Committee's  discussion.  The  attendees  review  included  the  following  (listed  by 
topic). 


Motions  to  Offer 


Number  of  Motions 

The  following  suggestions  were  made  regarding  the  number  of  motions  to  offer. 

•  One  motion  could  suffice,  as  Referral  Tracking  is  part  of  The  Plan. 

•  Make  two  motions: 

■=>     One  approving  the  2005-2007  Strategic  Plan  for  HIV  Prevention  Evaluation  in  SF  (The 

Plan),  and 
■=>    The  second  approving  the  proposed  Referral  Tracking  process. 
Alternative  two  motions: 

■=>     One  approving,  in  concept,  the  framework  of  The  Plan,  and 
o    The  second  approving  the  Objectives,  Strategies,  and  Action  Steps. 
Discussion  included  the  following. 
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Mike  Pendo  noted  that  rather  than  approving  either  The  Plan  or  Referral  Tracking,  the 
Committee  could  propose  the  Council  provide  " Approval  of concept." 

Kym  Dorman  said  that  the  Council  could  approve  a  framework  by  concept  vote. 
Israel  argued  that  Referral  Tracking  is  an  actual  shift  in  policy,  not  just  procedure,  and  so 
should  be  motioned  and  discussed  separately. 

o    Most  of  the  rest  of  The  Plan  is  reconfirming  what  is  already  done. 
Agreement 
The  members  agreed  to  make  two  motions. 


Referral  Tracking 


There  was  a  great  deal  of  discussion  of  the  importance  of  Referral  Tracking  as  well  as  some  of 
difficulties  the  topic  raises  to  providers,  including  the  following  highlights. 

Israel  noted  that  there  is  a  national  movement  to  evaluate  interventions. 

o    Not  all  are  interventions  are  to  be  evaluated  immediately,  rather  providers  should 
choose  which  are  to  be  reviewed  and  how. 

■=>    Criteria  are  different  for  different  interventions. 

Mike  P  noted  that  Referral  Tracking  was  attempted  several  years  ago  without  much  success, 

and  this  time  the  approach  needs  to  be  more  creative  so  as  to  involve  providers. 

Michael  D  suggested  explaining  that  the  most  important  thing  is  that  people  get  linked  with 

services,  and  that  to  do  so  efficiently  requires  some  tracking. 
Some  of  the  specific  issues  discussed  include  the  following. 

The  Word  "  Tracking" 

Several  members  expressed  concern  with  the  connotations  of  the  word, "  Tracking"and  several 

alternatives  were  raised  including: 

"=>     Linkage; 

■=>    Confirmation  of  Referrals; 

■=>    Documentation; 

<=>    Verification; 

<=>    Recording; 

■=>    Confirmation  of  Referral  (CR); 

o    Confirmation  and  Recording  of  Referrals  (CRR); 

■=>    Comprehensive  Confirmation  and  Recording  of  Referrals  (CCRR); 

o    Did  they  go  where  they  we  sent  them; 

<=>    Referral  Verification; 

>=>    Referral  and  Confirmation  of  Referral;  and 

<=>    Referral  Confirmation  and  Recording. 
Discussion  included  the  following. 

Allison  noted  that  other  jurisdictions  are  calling  it "  Tracking"  and  argued  for  the  value  of 

keeping  language  consistent. 

Michael  Discepola  said  that  part  of  the  Committee's  role  is  to  make  the  processes  easier 

and  friendlier  for  providers  thus  promoting  greater  participation. 

Mike  P  suggested  that  using  a  different  term  would  be  disingenuous,  and  that  providers  will 

see  through  it. 
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Perry  argued  the  importance  of  using  the  right  terminology  so  as  not  to  alienate  providers 

and  community  members. 

<=$     He  added  that  during  the  presentation  it  could  be  explained  that  what  is  being  referred 
to  is  Referral  Tracking 
Agreements/Action 

Agreement  was  reached  to  use  terminology  other  than, " Referral  Tracking"   Kym  will  circulate 
worth-smithing  ideas  to  members  my  email  09/03/04. 


Content  and  Order  of  Presentation 


The  following  changes  to  the  draft  presentation  were  agreed  upon. 
The  Plan 
<=>    The  flow  chart  (slide  9)  provides  an  overview  of  the  Goals  and  should  go  before  the 

Goals'  descriptions. 
<=>    Presenter  will  only  review  some  of  the  bullet-listed  Strategies  in  each  Objective. 
■=>    Perry  will  ask  members  to  hold  questions  to  the  end  of  the  review  of  each  goal. 
■=>     After  reviewing  The  Plan  the  presentation  should  offer  the  motion  to  approve. 
Referral  Tracking 
■=>    Referral  Tracking  will  be  discussed  during  the  overall  review  and,  in  greater  depth,  in 

association  with  the  motion  to  accept  the  Committee's  proposal  on  Referral  Tracking. 
o    Perry  will  note  that  Strategies  other  than  Referral  Tracking  may  also  have  policy  and/or 

RFP  implications  requiring  the  Council's  approval. 
Overall 

■=>    Perry  will  be  the  Presenter,  with  other  members  providing  support. 
o    Order  of  presentation  sections/slides: 

-      Title  Slide; 

Overview  of  presentation's  objectives 

*  Note  there  are  two  motions  for  Referral  Tracking  and  The  Plan; 
Visual  /  Chart  of  Goals 

*  As  introduction  to  The  Plan 
Review  of  The  Plan 

*  Including  some,  but  not  all,  highlighted  Strategies  within  each  Goal; 
Motion  to  approve  The  Plan;  and 

Discussion  &  Vote; 
Referral  Tracking  introduction; 
Explanation  of  Referral  Tracking; 
Committee's  proposal  regarding  Referral  Tracking; 
Motion  to  approve  proposed  Referral  Tracking  process;  and 
Discussion  &  Vote. 
<=o    It  was  noted  that  changes  may  be  offered  and  amendments  made  at  the  Council  meeting 
by  way  of  including  such  alterations  in  the  motion  to  approve. 
Re-draft  of  both  the  presentation  and  The  Plan  will  be  sent  to  members  09/03/04. 


HIV  PREVENTION  PLANNING  COUNCIL  (HPPC) 

Evaluation  Committee 

/Action  Minutes  From  Meeting: 

September  2,  2004 


4     Continue  Work  on  Strategic  Plan  for  Evaluation 

The  following  changes  to  The  Plan  were  proposed  and  discussed. 

Goal  One  - 

o     Strategy  1.3  A)  should  end  with  language  from  Strategy  1.3B), n  (and)  reflect 
community/program  priorities. " 

Goal  Two  - 

■=>     Abbreviate  /  make  shorter  and  simplify  the  wording  of  the  Strategies 

=>     Add  modifying  phrases,  such  as  " Conducted casually." 
•      Goal  Three  - 

■=>    Strategies  3.1  &  3.2  A)  add  n proven  practices"  in  the  list  of  "  Summary  of  findings  of 
intervention  studies" 

Agreement 

Motion  was  made  and  seconded  to  amend  The  Plan  as  discussion.  There  was  no  further 

discussion,  objection,  or  abstention.   The  motion  was  approved. 

Members  discussed  changing  the  wording  of  the  Action  Steps  throughout  The  Plan  as  the 

HPPC  doesn't  track,  but  can  be  part  of  dialogue,  and  may  request  updates  and/or  reports  on 

issues. 

Agreement 

The  Committee  agreed  that  the  wording  of  the  Action  Steps  should  reflects  the  HPPC's 

actual  role. 

<=>    The  Plan  submitted  for  approval  09/09/04  will  incorporate  this  understanding. 


Action  Steps 


Allison  said  that  the  items  listed  in  the  Agenda  are  those  Action  Steps  that  still  need  to  be 
reviewed  and  agreed  upon  by  the  Committee.   Discussion  followed. 

1.2  B)  Provide  technical  assistance  to  providers  to  fulfill  evaluation  requirements  ... 

Kym  suggested  the  Committee  agree  upon  a  definition  of  Technical  Assistance  (TA). 

Israel  said  that  this  Action  Step  was  intended  specifically  for  providers  who  are  not 

familiar  with  HIV  prevention  processes  and  procedures. 

Michael  D  said  that  this  Action  Step  more  accurately  should  be  that  the  HPPC  (not  the 

Evaluation  Committee)  recommends  that  the  AIDS  Office  provides  TA. 

Allison  noted  that  Compass  Point  did  a  Needs  Assessment  on  this  and  that  there  is  a 

process  in  place  to  identify  and  address  TA  issues. 

Allison  suggested  rather  than  recommending  processes,  the  HPPC  should  foster 

partnerships  with  providers. 

■=>    Michael  D  added  information  sharing  as  part  of  such  partnerships. 

Israel  suggested  the  provision  of  TA  f/oivin  response  to  the  previous  Strategies  in  Goal 

One  -  conducting  literature  review  (1.1  A)  and  research  on  interventions  (1.2  A). 

■=>    He  added  that  this  might  be  addressed  by  way  of  word-smithing. 

Agreements  Reached 

o    Change  Strategy  1.2  B): 

To  include,  "to  help  fulfill  requirements"; 
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Reword  "  HPPC will make  sure  that  providers  receive  TA  to  understand."  to 
n  Evaluation  Committee  recommends  that  the  HPPC  be  part  of  the  discussion 
regarding  how  TA  is  provided ' ..." 

1.3  A)  Make  evaluation  useful  and  meaningful. 

•      Kym  asked  if  this  Action  Step  should  be, "  The  Evaluation  Committee  recommends  HPPC  and 

the  AIDS  Office  be  in  regular  conversation  /  dialogue  with  providers  regarding  how  to  make 

evaluation  data  available  and  useful"  rather  than, "  The  Evaluation  Committee  recommends 

processes  for  how  HPPC  will  collaborate  ..." 

Allison  suggested  the  Committee  recommend  an  inventory  of  data  from  the  ELI  system. 

o    Israel  noted  that  such  is  part  of  Marise  Rodriguez's  presentation, "  Client  Level  Data 
Collection  Systems"  which  was  moved  from  the  08/12/04  to  the  09/09/04  Council 
meeting  agenda. 

=>     He  added  that  this  topic  is  on  the  Parking  Lot. 

Agreements  Reached 

It  was  agreed  to  accept  Kym's  recommendation  for  restating  Action  Step  1.3  A). 


5.    Next  Steps,  Evaluation,  and  Closure 

At  the  next  meeting  10/07/04  discussion  will  continue  on  the  Action  Steps  on  the  agenda, 
but  not  covered  at  this  meeting: 

1.4  A)     Development  of  standardized  outcomes/performance  measures. 

2.1  A)     Use  of  CDC's  performance  indicators. 

Motion  to  adjourn  was  offered  and  seconded.  There  was  no  discussion,  or  objection.  The 
meeting  adjourned  at  5:03  PM. 


THE  NEXT  MEETING  IS  SCHEDULED  FOR  THURSDAY,   OCTOBER  7" ,  2004 
FROM  3:00  TO  5:00  PM  -  25  Van  NESS  A  VENUE,  SUITE  330B. 


Mi 


nutes  were  prepared  by  David  Weinman,  Reviewed  by  Lisa  Reyes  and  Perry  Rhodes  III. 
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1.  Welcome  and  Announcements  3:00-3:10 

2.  Public  Comment  3:10-3:20 

3.  Approve  Minutes  from  September  2,  2004  3:20-3:25 

4.  Continue  Work  on  Strategic  Plan  for  Evaluation:  3:25-4:00 
Complete  Action  Steps 

5.  Referral  System:   Discuss  and  Finalize  Methods  for  4:00-4:50 
Evaluating  Successful  Referrals 
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NOTE:  All  meetings  are  open  to  the  public  and  are  held  In  handicapped  accessible  facilities. 
Meeting  dates  and  times  are  subject  to  change,  please  verify  by  calling  Betty  Chan  Lew  at  554-9492. 

Know  your  rights  under  the  Sunshine  Ordinance;  Government's  duty  is  to  serve  the  public,  reaching  its  decisions  In  full  view  of  the 
public.  Commissions,  boards,  councils,  and  other  agencies  of  the  city  and  county  exist  to  conduct  the  people's  business.   This 
ordinance  assures  that  deliberations  are  conducted  before  the  people  and  that  City  operations  are  open  to  the  people's  review.  For 
more  information  on  your  rights  under  the  sunshine  ordinance  or  to  report  a  violation  of  the  ordinance,  contact  the  Sunshine 
Ordinance  Task  Force.   Donna  Hall,  City  Hall,  Room  244,  1  Dr.  Cariton  B.  Goodlett  Place,  San  Francisco,  CA  941 02,  Phone;  554- 
7724,  Fax;  554-7654,  E-Mail:  Donna_Hall@ci.sf.ca.us- 

In  order  to  assist  with  the  City's  effons  to  accommodate  persons  with  severe  allergies,  environmental  illnesses,  multiple  chemical 
sensitivity  or  related  disabilities,  attendees  at  public  meetings  are  reminded  that  other  attendees  may  be  sensitive  to  vanous 
chemical  based  products.  Please  help  the  City  accommodate  these  Individuals. 
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Members  Present:  Marcel  Miranda,  Israel  Nieves-Rivera,  Perry  Rhodes,  III,  Lisa  Reyes  and  Frank 
Strona. 

Members  Absent:  Michael  Discepola  and  Chata  Ashley. 

Professional  Staff:  Kym  Dorman  (Harder  &  Co),  Tracey  Packer  (HPS),  Mike  Pendo  (HPS),  and  David 
Weinman  (Note  Taker). 

Guests:  Janetta  Johnson  (Member  HPPC),  Giovanni  Petruccelli  (Haight-Ashbury  Preventative 
Services),  Dina  Wilderson  (Larkin  Street),  and  Teddy  Partridge  (Stop  AIDS) 


Welcome  and  Announcements 

Perry  Rhodes,  III  called  the  meeting  to  order  at  3:13  PM.   He  invited  attendees  to  introduce 
themselves  to  make  announcements. 

Perry  announced  a  fundraising  event  10/16/04  for  Cal  Pep  in  Oakland.   He  will  email 

announcements  to  members. 

Israel  Nieves-Rivera  noted  that  10/15/04  is  National  Latino  AIDS  Day. 

•  Israel  also  announced  that  a  video  about  AIDS  in  the  Latino  community  has  been  locally 
produced  and  that  about  500  copies  are  available  for  agencies  serving  Latino  populations. 

•  Frank  Strona  announced  a  Community  Forum  entitled,  "Multiple  Lives:  a  Man's  Personal 
Journe/  will  be  held  10/27/04  at  the  Main  Branch,  SF  Public  Library. 

o    The  purpose  of  the  forum  is  to  de-stigmatize  men  on  the  "down-low." 

o     He  noted  this  is  a  collaboration  between  Stop  AIDS  and  the  SF  DPH. 

o    Participants  from  Alameda  County,  and  other  surrounding  areas,  would  be  welcomed. 

■=>     He  also  pointed  out  that  men  of  color  face  far  more  stigma  than  white  men. 


Public  Comment 

There  was  no  public  comment. 


3.    Approval  of  Minutes  from  September  2,  2004 

The  following  corrections  were  noted: 

o    The  date  on  the  document  reads,  "August  5,  2004,  "should  be,  "September  2,  2004;' 
o    Tracey  Packer  was  listed  as  in  attendance,  but  was  in  fact  not  at  this  meeting. 

Motion  was  made  and  seconded  to  accept  the  Minutes  as  amended.   There  as  no  further 

discussion,  or  objection.  The  minutes  were  approved  with  Marcel  Miranda  and  Frank  Strona 

abstaining. 
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Continue  Work  on  Strategic  Plan  for  Evaluation:  Complete  Action  Steps 

Kym  Dorman  distributed  the  document  entitled, " Changing  the  Culture  of  Evaluation  in  SF  HIV 
Prevention  2005-2007  strategic  PLAN  FOR  HIV  PREVENTION  evaluation  IN  5F  -  Draft  October  7, 
2004'  (the  Strategic  Plan),  a  copy  of  which  is  available  to  absent  members  upon  request.  Israel 
Nieves-Rivera  provided  some  background  particularly  for  the  guests. 

o    In  2003  HPPC  rewrote  its  multi-year  HIV  Prevention  Plan  (The  Plan). 

As  a  result  the  Evaluation  Committee  was  formed  for  the  Council's  2004  term. 

The  Evaluation  Committee's  charge  has  been  to  come  up  with  ar\  evaluation  strategy  that 

adheres  to  The  Plan's  principles  and  is  comprehensive,  applicable  citywide,  and  not 

program  specific. 

The  drafted  Strategic  Plan  provides  Goals,  Objectives,  and  Strategies  to  address  SF's 

Evaluation  need,  particularly  in  light  of  the  CDC's  new  initiative  (^Advancing  HIV 

Prevention"). 

The  draft  also  provides  an  overview  of  who  is  responsible  for/participates  in 

accomplishing  each  of  the  components  of  the  Strategic  Plan. 

This  is  a  strategic  approach  to  Evaluation  based  on  what  the  Community's  objectives. 

The  Strategic  Plan  is  comprised  the  three  Goals  (page  6): 
Goal  1  -  Gathering  the  Stories; 
Goal  2  -  Creating  Information  Bridges;  and 
Goal  3  -  Using  the  Stories  to  Create  Change. 
Discussion  followed,  including  the  following  highlights. 

General  /  Wording 

Marcel  said  that  the  Introduction  reads  awkwardly. 

<=>    Teddy  Partridge  offered  that  it  reads  somewhat  awkwardly,  but  understandably  so. 

<=>    Kym  will  review  and  make  adjustments  /  word-smithing. 
Kym  highlighted  a  number  of  instances  where  the  draft  reads, " HPPC in  collaboration  with  the 
AIDS  Office  should  be  part  of  the  discussion  to."  She  suggested  and  there  was  general 
agreement  that  the  Committee  review  each  occurrence,  as  follows. 


Strategy- 


•      Discussion 
Agreement 


1.1  A) 


Wording  okay  as  is. 


1.2  A) 


Frank  suggested  adding  a  chart  showing  the  current  status  of  each  item 

including  that  activities  are  " in  process"  or  "planned'. 

■=>    He  added  that  the  chart  could  be  an  appendix  to  the  Strategic  Plan. 

Israel  said  that  members  would  probably  know  what  the  statement 

means,  but  that  it  might  not  be  clear  to  other  readers. 

Marcel  said  that "  should  be  part  of  the  discussion"  is  not  strong  enough, 

but  while  the  Committee  doesn't  want  to  dictate,  it  should  be  clear  that 

the  work  between  the  HPPC  and  the  AIDS  Office  is  a  process. 

Tracey  suggested  that  this  Strategy  read  "HPPC in  collaboration  with 

the  AIDS  Office  should  prioritize..." 

Israel  asked  if  the  term,  "research"  is  comprehensive  enough, 
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•      Discussion 
Agreement 

questioning  if  it  includes  Needs  Assessments  and  other  small  projects. 
■=>    He  suggested  what  ^research "includes  should  it  be  specified. 
Marcel  said  the  Committee  shouldn't  define  the  HPPC's  role  too  narrowly. 
■=>     He  said  that  the  HPPC  has  a  responsibility  to  point  out  if  it  sees 
something  being  overlooked. 

•  Lisa  Reyes  said  that  in  this  Strategy  research  is  on  interventions  which 
defines  the  research  narrowly  enough. 

•  Tracey  suggested  wording  that  could  include  Needs  Assessment  and  the 
like,  adding  that  the  intent  is  to  keep  current  /  up  to  date  on  what  is 
being  done. 

Israel  suggested  adding  some  specifics  in  the  Action  Steps. 

•  He  also  suggested  adding  working  with  providers. 

•  Teddy  Partridge  suggested  adding  footnotes  specifying  requirements. 

•  Tracey  suggested  "  The  HPPC  in  collaboration  with  the  AIDS  Office 
iden  tifies  priorities  and  recommends. . . " 

It  was  agreed  that  the  Recommendation  read,  "The  HPPC  in  collaboration 
with  the  AIDS  Office  identifies  priorities  and  recommends...  " 

1.3  B) 

Same  as  1.2  B) 

1.3  C) 

•  Kym  suggested  adding  this  to  the  chart  Frank  suggested  earlier  -  a 
snapshot  of  how  qualitative  research  is  currently  being  handled. 

^    She  observed  that  a  visual  presentation  makes  it  more  interesting. 
■=>     She  added  that  the  chart  should  highlight  that  it  can/will  change. 
Israel  suggested  using  HPPC/DPH/HPS  activities  in  Visitation  Valley  and 
the  Gay  Men's  Health  Initiative,  as  examples  of  qualitative  research. 

•  He  also  suggested  that  the  qualitative  research  process  should  include 
provisions  for  the  findings  to  be  reported  back. 

•  Mike  Pendo  suggested  the  examples  should  also  note  that  there  is  a  lot 
of  similar  qualitative  research  underway. 

•  Frank  suggested  changing  the  wording  to, "...  will  collaborate  with  the 
AIDS  office  to  identify  interventions..." 

Change  wording  to,  "  The  HPPC  will  collaborate  with  the  AIDS  Office  to 
identify  interventions... " 

1.4  A) 

Change  wording  to,  "  The  HPPC  will  collaborate  with  the  AIDS  Office." 
(shorter  and  simpler) 

2.1  B) 

It  was  suggested  to  adjust  the  wording  as  per  previous  items. 
.      It  was  also  suggested  to  add  that  findings  of  programs'  effectiveness 
will  be  reported  back  to  the  HPPC 

Marcel  said  that  it  isn't  clear  to  an  outsider  who  is  determining  program 
effectiveness  and/or  taking  action  on  such  findings. 
Israel  observed  that  determining  program  effectiveness  isn't  being  done 
this  year,  but  will  happen  in  future  years. 
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<=>     He  added  that  the  information  is  currently  being  gathered,  but 

findings  are  not  yet  available. 
Mike  suggested  changing  the  wording  from  "Use  qualitative  findings..."  to 
"Use  qualitative  data..." 
■=>    He  added  that  it  needs  to  be  clear  that  the  Strategic  Plan  supports 

qualitative  data. 
Kym  noted  that  this  Strategy  links  to  Strategies  in  Goal  3. 
Change  wording  to,  "Use  qualitative  data  to  determine..." 

2.2  A) 

Adjust  language  as  per  wording  in  previous  Strategies,  and  change  language 
from  "...  identify  priority  emerging  community  issues,"  to  "identify 
prioritized  emerging  issues." 

2.3  A) 

•  It  was  suggested  the  language  should  read, "  The  HPPC  and  AIDS  Office 
works  together  to  share  information." 

•  Mike  said  that  Counseling  and  Testing  (CAT)  information  is  already 
shared,  but  it  is  not  very  clear  nor  is  it  shared  with  everyone. 

^    He  suggested  making  CAT  information  user  friendly  and  help 

providers  understand  the  available  data. 
■=>    Kym  added  making  the  information  is  accessible,  useful,  and  includes 

data  on  the  trends. 
<=>    It  was  also  suggested  that  this  Strategy  be  re-worded  to  "enhance," 

not  "create." 
■=>    Perry  suggested  this  Strategy  include  putting  CAT  data  on  the  HPS' 

web  site. 
It  was  agreed  to  reword  the  Strategy  and  Recommendations  as  per  the 
suggestions  and  in  accord  with  previous  Strategies. 

2.3  B) 

It  was  noted  that  information  is  being  collected,  although  currently  not 
being  used. 

•  Tracey  observed  that  the  Committee  has  indicated  that  it  wants  this  to 
be  a  priority. 

o    She  added  that  he  HPPC/HPS  will  figure  out  how  to  make  it  happen. 

•  Marcel  noted  that  agencies  are  collecting  and  reporting  information  on 
programs'  successes  and  ways  to  improve  them.   The  challenge,  he  added 
is  how  to  report  this  to  the  community. 

■=>    He  also  noted  that  that  while  this  information  is  reported  on  the 
monitoring  report,  agencies  aren't  given  "points"  for  tracking  this. 

<=>    Exploring  the  successes  and  finding  ways  to  improve  programs  is  the 
link  between  'Telling  the  story'  and  Evaluation. 

Israel  said  that  there  is  a  bridge  from  providers  to  HPS  but  the  story 

of  successes  is  not  being  shared. 

o    He  suggested  that  the  information  might  not  be  collected  in  a  way 
that  makes  sense  to  providers. 
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Kym  said  that  part  of  what's  missing  is  reporting  from  HPS  to  the 

community. 
•      Tracey  said  the  objective  should  be  to  do  this  better  and  suggested 

keeping  this  as  a  priority  with  the  understanding  that  how  to  do  it  is  an 

Action  Item  for  2005. 
It  was  agreed  to  discuss  this  Strategy  in  more  detail  at  the  December 
Committee  meeting  (12/02/04). 


3.1  &  3.2  A) 


It  was  agreed  to  reword  the  Recommendation  in  accord  with  previous  items. 
It  was  agreed  to  reword  the  Recommendation  in  accord  with  previous  items. 
It  was  agreed  to  reword  the  Recommendation  in  accord  with  previous  items. 


3.3  A) 


3.4  A) 


Kym  reminded  members  that  some  items  were  prioritized  for  early  discussion  so  they  could  be 
included  in  the  RFP. 

Other  Items/Suggestions 

Mike  Pendo  stated  noted  that  the  Strategic  Plan  shouldn't  make  specific  program 
improvement  specifications,  but  rather  focus  on  the  big  picture. 
•      Israel  and  others  said  that  many  components  of  the  Strategic  Plan  may  require  having  a 
standing  Evaluation  Committee;  particularly  to  report  back  findings  and  take  action  based  on 
those  findings. 
o    Tracey  suggested  that  the  Committee  not  make  specific  recommendations  regarding 

how  to  accomplish  what  the  Strategic  Plan  calls  for. 

She  added  that  the  Committee's  task  is  to  specify  what  needs  to  be  done. 

She  also  suggested  that  if  the  Committee  chooses,  it  should  make  a  separate 

recommendation  regarding  making  the  Evaluation  Committee  a  standing  committee. 
Teddy  Partridge  suggested  adding  a  column  showing  who  is  the  consumer  of  the 
recommended  Action  Steps. 


5.    Referral  System:    Discuss  and  Finalize  Methods  for  Evaluating  Successful  Referrals 

Perry  explained  that  the  purpose  of  the  discussion  is  to  develop  a  menu  of  methodology  options 
providers  could  choose  from  as  discussed  at  previous  Committee  meetings.  It  was  noted  that 
providers  would  choose  one  intervention  to  track  for  one  year. 
Methodology  Menu  Options: 

o    Referral  Cards,  follow-up  later 

*  Self  Addressed 

*  Color  Coded; 

o    Physically  accompanying  client  to  agency; 
■=>     Referral  Tracking  Form  -  with  follow-up;  and 
o     Self  Reporting 
Discussion  following  including  the  following. 
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Providers  should  be  asked  to  use  one  of  the  methods  listed,  or  they  may  use  their  own 

method  of  tracking. 

The  process  is  not  to  be  punitive,  and  if  it  doesn't  work  providers  should  try  something  else. 

Frank  suggested  combining  all  of  the  agencies  using  the  same  method  in  meeting  to  share 

information  on  that  methodology. 

<=>     Marcel  added  that  providers  who  use  the  same  methodology  could  evolve  into 

collaborating  between  agencies. 
=>     He  also  noted  that  currently  while  Memorandum  of  Understanding  (MOU)  are 

encouraged,  they  are  not  funded. 
!=>    Israel  said  that  connection  between  agencies  fits  into  agencies  logic  model. 
■=>     Frank  noted  that  agencies  working  on  the  same  methodologies  would  encourage 

collaboration  and  bring  some  clarity  and  priority  to  the  MOU  process. 
■=>    Israel  noted  that  this  would  be  useful  to  encouraging  agencies  to  meet  the  needs  of 

their  target  population(s). 
Giovanni  Petruccelli  asked  would  the  Referral  Tracking  being  proposed  would  apply  to  all 
agencies  a  client  goes  to  or  just  the  initial  referral. 
=*     Frank  responded  that  how  it  is  applied  would  be  based  on  how  the  various  agencies 

develop  it. 
Israel  noted  that  a  difficulty  has  long  been  encouraging  referral  to  non-prevention  agencies. 
Mike  P  said  that  the  kinds  of  referrals  should  be  self-generated,  that  if  the  HPS 
encourages  specifics  it  tends  to  become  a  sort  rule  which  would  not  produce  the 
collaborative  results  the  Strategic  Plan  envisions. 

Teddy  Partridge  suggested  that  self-generated  collaboration  doesn't  work  and  it  needs  to 
be  contractual  and  with  monetary  implications.. 

Tracey  added  that  the  need  is  for  the  development  of  linkages  that  work  for  the  clients. 
Frank  suggested  that  whoever  writes  this  part  of  the  RFP  needs  to  find  the  balance 
between  requiring  linkages  and  encouraging  self -generated  collaboration. 
Teddy  Partridge  suggested  there  is  a  need  for  a  structured  system  and  that  incentives 
should  be  built  into  the  system  as  part  of  the  overall  budget. 

Giovanni  Petruccelli  noted  that  clients  often  experience  that  agencies  are  unaware  of  each 
other. 

Perry  said  that  MOUs  aren't  always  practical,  that  they  can  get  too  big  and  complex,  and 
they  need  to  be  limited  so  as  to  be  manageable. 

Mike  said  that  the  first  step  is  to  determine  if  a  referral  has  taken  place  and  that  the 
Council  and  providers  can  work  from  there  on  effectiveness. 
Perry  suggested  starting  small. 

Frank  suggested  the  menu  include  what  kind  of  referral  and  to  whom. 
=*     He  added  that  who  referrals  are  made  to  should  be  from  within  the  providers'  list  of 

existing  MOUs  with  an  emphasis  on  those  collaborations  that  make  the  most  sense. 
Israel  added  that  the  idea  is  to  understand  who  the  provider's  pariners  are. 
Teddy  Partridge  suggested  calling  this  a  "pilot  program." 
Marcel  suggested  a  guideline  such  as  10%  of  a  provider's  clients. 
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Tracey  suggested  a  guideline  of  a  percentage  of  a  program  or  an  intervention  whichever  is 
more  practical 
•      Perry  suggested  he,  Kym,  Tracey  and/or  Mike  get  together  and  put  together  a  form  for  the 
members'  consideration. 


6.    Next  Steps,  Evaluation,  and  Closure 

At  the  next  meeting  12/02/04  discussion  will  continue  on  the  Action  Steps  on  the  agenda,  but 
not  covered  at  this  meeting: 

Motion  to  adjourn  was  offered  and  seconded.   There  was  no  discussion,  or  objection.  The 
meeting  adjourned  at  5:03  PM. 


THE  NEXT  MEETING  IS  SCHEDULED  FOR  THURSDAY,  DECEMBER?0,  2004 
FROM  3:00  TO  5:00  PM  -  25  VAN  NESS  A  VENUE,  SUITE  330B. 


Minutes  were  prepared  by  David  Weinman,  Reviewed  by  Tracey  Packer,  Lisa  Reyes  and  Perry  Rhodes 
III. 


,1-24-2004      15:10 
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JtiWf  PREVENTION  PLANNING  COUNCIL  (HPPC) 
^  Evaluation  Committee 
Thursday,  December  2,  2004 

3:00  pm  -  5:00  pm 
25  Van  Ness  Avenue,  330B 
^■San  Francisco 


AGENDA 


1.    Welcome  and  Announcements 


2.  Public  Comment 


3:00-3:10 
3:10-3:20 


3.   Approve  Minutes  from  October  7,  2004 


3:20-3:25 


4.   Report  from  Steering  Committee 


3:25-3:35 


5.   Review  Strategic  Plan  for  Approval  (vote) 


3:35-4:15 


6.   Discuss  Next  Steps  for  2005 


4:15-4:50 


7.   Next  Steps.  Evaluation,  and  Closure 


4:50-5:00 
DOCUMENTS  DEPT. 
NOV  2  4  2004 

SAN  FRANCISCO 
PUBLIC  LIBRARY 

1 1-24-0  4 P04: 01     RCVD 


NOTE:  All  meetings  are  open  to  the  public  and  are  held  In  handicapped  accessible  facilities. 
Meeting  dates  and  times  are  subject  to  change,  please  verify  by  calling  Betty  Chan  Law  at  554-9492. 

Know  your  rights  under  the  Sunshine  Ordinance:  Government's  duly  is  to  serve  the  public,  reaching  lt$  decisions  in  full  view  of  the 
public.   Commissions,  boards,  councils,  and  other  agences  of  the  city  and  county  exist  to  conduct  the  people's  business.   This 
ordinance  assures  that  deliberations  are  conducted  before  the  people  and  that  City  operations  are  open  to  the  people's  review.  For 
more  Information  on  your  rights  under  the  sunshine  ordinance  or  to  report  a  violation  of  the  ordinance,  contact  the  Sunshine 
Ordinance  Task  Force    Donna  Hall,  City  Hall,  Room  244.  1  Dr.  Carlton  B  Goodiett  Place,  San  Francisco,  CA  94 102,  Phone:  554- 
7724.  Fax:  554-7854.  E-Mail;  Donna_HaH@Ci.sf.ca.uS. 

In  order  to  a$$i$t  with  the  City's  efforts  to  accommodate  persons  with  severe  allergies,  environmental  illnesses,  multiple  chemical 
sensitivity  or  related  disabilities,  attendees  at  puoilc  meetings  are  reminded  that  other  attendees  may  be  sensitive  to  various 
chemical  based  products    Please  help  the  City  accommodate  these  individuals. 
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Members  Present:  Marcel  Miranda,  and  Perry  Rhodes,  III. 

Members  Absent:  Michael  Discepola,  Israel  Nieves-Rivera,  Lisa  Reyes  and  Frank  Strona. 

Professional  Staff:  Kym  Dorman  (Harder  &  Co),  Tracey  Packer  (HPS),  and  David  Weinman  (Note 
Taker). 


Welcome  and  Announcements 

Perry  Rhodes,  III  called  the  meeting  to  order  at  3:15  PM.   He  observed  that  this  was  his  first 
year  of  involvement  with  the  HPPC  and  expressed  satisfaction  with  what  he  learned,  observed, 
and  gained  from  the  experience.   Discussion  followed  about  the  value  and  benefits  of  the 
community  planning  process. 


2.    Public  Comment 

There  was  no  public  comment. 


3.    Approval  of  Minutes  from  October  7,  2004 

As  there  was  no  quorum  the  minutes  could  not  be  approved,  however,  no  additions,  corrections 
or  changes  were  offered. 


4.    Report  from  Steering  Committee 

There  was  no  report  from  the  Steering  Committee. 


Review  Strategic  Plan  for  Approval 

Kym  Dorman  noted  that  a  draft  of  the  Strategic  Plan  entitled,  "Change  the  Culture  of 
Evaluation  in  5F  HIV  Prevention"  (dated  11/23/04)  was  distributed  to  members  by  e-mail.   She 
distributed  a  final  draft  of  the  Strategic  Plan  (dated  12/02/04),  a  copy  of  which  is  available  to 
absent  members  upon  request.   Kym  explained  that  the  changes  discussed  and  agreed  upon  are 
reflected  and  highlighted  in  both  drafts  (11/23  and  12/02/04).   Discussion  followed. 

.      Kym  drew  the  members'  attention  to  the  changes  in  the  penultimate  paragraph  on  page  1 
(beginning, "  The  purpose  of  this  strategic  plan... ')  of  the  12/02/04  draft. 
■=>    The  last  three  sentences  of  that  paragraph  have  been  re-written  for  clarity's  sake. 
Perry  asked  about  the  HPPC  monitoring  items  that  are  to  be  implemented  by  others. 
■=>    Tracey  Packer  suggested  that  implementation,  including  monitoring,  be  included  in  the 

items  to  be  addressed  by  next  year's  successor  committee  (the  '05  Committee). 
Strategy  1.1  A)  Action  Steps  (page  7) 
■=>    Marcel  Miranda  suggested  that  the  " experts"  called  for  be  defined. 
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Kym  cited  Willi  McFarland  and  others  as  examples  of  experts. 

There  was  general  agreement  on  this  clarification. 
Tracey  pointed  out  that  the  second  bullet's  reference  to  the, " last  consensus  meeting" 
may  not  be  appropriate  because  consensus  meetings  deal  with  incidence  and  prevalence 
rather  than  intervention. 

Kym  explained  that  the  consensus  meetings  were  cited  because  their  findings 

reflect  the  effectiveness  of  citywide  activities  /  interventions. 

Marcel  added  that  the  incidence  and  prevalence  numbers  show  the  Council  where 

the  epidemic  is  and  what  activities  make  sense. 

Tracey  suggested  the  reference  be  changed  to  include  epidemiologic  information. 

There  was  agreement  on  this  change. 
Kym  noted  an  overall  gap  in  qualitative  information. 
■=>    It  was  suggested  that  this  be  part  of  the  '05  Committee's  objective. 
Strategy  1.2  B)  Action  Steps  (page  8) 
o     Marcel  noted  the  wording  is  a  run-one  sentence. 
<=>     Kym  suggested  adding  parenthesis  around  the  clause,  "who  receive  awards  before 

contracts  are  approved'  to  address  the  grammar. 
Strategy  1.3  A)  Action  Steps  and  Status  of  Strategy  (page  8) 
■=>    Marcel  noted  the  typo  which  reads, " ...Ob/TA  were.."  which  should  read  "  ...Ob/TA 

where."  (as  does  1.3  B). 
Strategy  1.3  B)  Action  Steps  (page  8) 

■=>    Marcel  suggested  changing  the  to,  "...outcomes  and  to  suggest  new  outcomes" 
Strategies  2.2  A),  B)  Action  Steps  (page  10) 
o    Marcel  suggested  removing  "appropriate!'  from  the  clause,  "all appropriate  community 

stakeholders." 
■=>    There  was  general  agreement  on  this  change. 

It  was  noted  that  the  1997  Plan  had  a  good  strategic  plan  for  evaluation,  but  that  national 
emphasis  on  client-level  data  collection  took  priority. 

Kym  asked  about  the  mechanism  of  taking  the  final  draft  of  the  Strategic  Plan  to  the  HPPC 
for  review/approval. 
<=>     Discussion  followed  including  - 

The  Council  approved  the  Strategic  Plan  as  submitted  by  the  '04  Evaluation 

Committee  including  the  Strategic  Plan,  the  Goals,  Objectives,  and  Strategies. 

'05  Committee  will  submit  the  final  to  HPPC,  including  their  recommendations 

regarding  the  issues/components  they  are  charged  with. 


6.    Discuss  Next  Steps  for  2005 

The  following  are  the  items  discussed  as  recommendations  for  the  '05  Committee. 
•      Action  Steps 

^    It  was  explained  that  the  Council  has  not  as  yet  approved  the  version  of  the  Action 
Steps  included  in  the  Strategic  Plan. 
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o  It  was  noted  that  the  substance  of  the  Action  Steps  had  previously  been  discussed  and 
agreed  upon  and  that  the  recommended  changes  are  not  substantive  -  consisting  mostly 
of  grammar. 

<=>    It  was  agreed  that  completion  of  the  Action  Steps  and  their  submission  to  the  Council  is 
a  task  for  the  '05  Committee. 

•  Implementation  of  the  Strategic  Plan 

<=>    -How  and  when  should  the  HPPC  do  things 

Methodology,  including: 

<=>     How  should  the  DPH/AIDS  Office  and/or  others  accomplish  the  actions  called  for? 

o    Reports  back  from  DPH/AIDS  Office  to  HPPC  about  what  has  and  hasn't  been  done. 

Perhaps  regular  (annual)  reports  to  HPPC  on  evaluation  activities  by  Strategy, 
o    Incorporate  progress  on  evaluations  as  addendum  to  The  Plan. 
A  Timeline  showing  when  things  are  to  be  implemented, 
o    Including  reviewing  the  Action  Steps  and  submitting  them  to  the  Council. 
Provide  a  Forum  for  the  community,  DPH,  and  providers  to  discuss  evaluation. 
<=>    Committee  meetings  are  such  a  forum,  are  they  enough? 

•  Identify  the  gaps  in  Qualitative  Data  and  proposing  means  of  address  the  lack  of  data. 
Explore  and  explain  the  real  Purpose  of  Evaluation. 

<=>    It  should  not  be  looked  at  as  punitive. 

o    It  was  noted  that  the  new  RFP  has  components  from  Strategic  Plan,  which  is  a  good 

start  to  addressing  this  topic. 
^     Discussion  followed  about  the  meaning,  purpose,  and  culture  of  evaluation. 

The  Culture  of  Evaluation 

Tracey  said  that  part  of  the  HPPC  and  AIDS  Office's  purpose  should  be  to  change  the 
culture  of  evaluation  and  that  this  Strategic  Plan  is  part  of  way  of  addressing  this. 
Perry  noted  the  diversity  of  the  communities  dealing  with  prevention  makes  a  consistent 
view  of  evaluation  difficult. 
■=>    The  Providers'  Contract  Managers  and  the  DPH's  Evaluation  Counselors  need  to  routinely 

work  together  on  evaluation. 
Tracey  explained  that  a  current  reorganization  within  the  AIDS  Office  is  partly  motivated 
by  the  department's  need  to  emphasize  a  new  culture  for  evaluation. 
Marcel  suggested  producing  an  explanatory  brochure  about  evaluation  similar  to  the 
Condensed  Plan  for  2001;  although  perhaps  not  as  elaborate. 

He  explained  that  this  was  a  glossy,  oversized,  magazine-like  brochure. 
<=>    Such  a  brochure  should  highlight  evaluation's  value  to  both  providers  and  the  community. 
■=>    He  added  that  it  should  also  be  accessible/understandable  to  providers  at  all  levels  of 

technical  expertise. 
Kym  suggested  ensuring  the  'Culture  of  Evaluation  is  brought  up  at  all  community  forums. 
o    She  added  that  when  it  is  discussed  in  such  a  setting  the  explanation  should  be  easily 

understood  and  friendly. 
Perry  suggested  that  what  evaluation  really  is  needs  to  be  explained;  that  it  is  a  way  to 
ensure  that  providers'  approach  is  working  and  to  show  who  is  being  reached. 
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Next  Steps,  Evaluation,  and  Closure 

Perry  said  that  the  Committee  needs  an  end  point  and  questioned  wrapping  up  the  year's  work 
without  a  quorum.  Tracey  explained  that  the  assembled  members  are  empowered  to  close  the 
term.   It  was  agreed  that  the  2004  Evaluation  Committee's  work  was  completed. 


The  meeting  adjourned  at  4:25  PM. 


Minutes  were  prepared  by  David  Weinman,  Reviewed  by  Tracey  Packer  and  Perry  Rhodes  III. 


"'■•■■*.' 


■r  i 


mm. 


ma/? 


''■"■■■■ 


,  ■■  ■* 


-,*,:^'' '-• 


Mj 


BE 


»••  M 


